diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..52373fe24473b1aa44333d318f578ae6bf04b49b 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/README.md b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..e47dac806028aa12c2b6f5e97b882d37ec7f9bf6
--- /dev/null
+++ b/README.md
@@ -0,0 +1,182 @@
+---
+tags:
+- unsloth
+base_model:
+- zai-org/GLM-4.6V
+language:
+- zh
+- en
+library_name: transformers
+license: mit
+pipeline_tag: image-text-to-text
+---
+> [!NOTE]
+>  Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
+>
+
+<div>
+<p style="margin-top: 0;margin-bottom: 0;">
+    <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
+  </p>
+  <div style="display: flex; gap: 5px; align-items: center; ">
+    <a href="https://github.com/unslothai/unsloth/">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
+    </a>
+    <a href="https://discord.gg/unsloth">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
+    </a>
+    <a href="https://docs.unsloth.ai/">
+      <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
+    </a>
+  </div>
+</div>
+
+
+# GLM-4.6V
+
+<div align="center">
+<img src=https://raw.githubusercontent.com/zai-org/GLM-V/refs/heads/main/resources/logo.svg width="40%"/>
+</div>
+
+This model is part of the GLM-V family of models, introduced in the paper [GLM-4.1V-Thinking and GLM-4.5V: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://huggingface.co/papers/2507.01006).
+
+-   **GLM-4.6V Blog**: [https://z.ai/blog/glm-4.6v](https://z.ai/blog/glm-4.6v)
+-   **Paper**: [https://huggingface.co/papers/2507.01006](https://huggingface.co/papers/2507.01006)
+-   **GitHub Repository**: [https://github.com/zai-org/GLM-V](https://github.com/zai-org/GLM-V)
+-   **Online Demo**: [https://chat.z.ai/](https://chat.z.ai/)
+-   **API Access**: [Z.ai Open Platform](https://docs.z.ai/guides/vlm/glm-4.6v)
+-   **Desktop Assistant App**: [https://huggingface.co/spaces/zai-org/GLM-4.5V-Demo-App](https://huggingface.co/spaces/zai-org/GLM-4.5V-Demo-App)
+
+## Introduction
+
+GLM-4.6V series model includes two versions: GLM-4.6V (106B), a foundation model designed for cloud and high-performance
+cluster scenarios,
+and GLM-4.6V-Flash (9B), a lightweight model optimized for local deployment and low-latency applications.
+GLM-4.6V scales its context window to 128k tokens in training,
+and achieves SoTA performance in visual understanding among models of similar parameter scales.
+Crucially, we integrate native Function Calling capabilities for the first time.
+This effectively bridges the gap between "visual perception" and "executable action"
+providing a unified technical foundation for multimodal agents in real-world business scenarios.
+
+![GLM-4.6V Benchmarks](https://raw.githubusercontent.com/zai-org/GLM-V/refs/heads/main/resources/bench_46v.jpeg)
+
+Beyond achieves SoTA performance across major multimodal benchmarks at comparable model scales. GLM-4.6V introduces
+several key features:
+
+- **Native Multimodal Function Calling** 
+Enables native vision-driven tool use. Images, screenshots, and document pages can be passed directly as tool inputs without text conversion, while visual outputs (charts, search images, rendered pages) are interpreted and integrated into the reasoning chain. This closes the loop from perception to understanding to execution.
+
+- **Interleaved Image-Text Content Generation**
+Supports high-quality mixed media creation from complex multimodal inputs. GLM-4.6V takes a multimodal context—spanning documents, user inputs, and tool-retrieved images—and synthesizes coherent, interleaved image-text content tailored to the task. During generation it can actively call search and retrieval tools to gather and curate additional text and visuals, producing rich, visually grounded content.
+
+
+- **Multimodal Document Understanding**
+GLM-4.6V can process up to 128K tokens of multi-document or long-document input, directly interpreting richly formatted pages as images. It understands text, layout, charts, tables, and figures jointly, enabling accurate comprehension of complex, image-heavy documents without requiring prior conversion to plain text.
+    
+- **Frontend Replication & Visual Editing** 
+Reconstructs pixel-accurate HTML/CSS from UI screenshots and supports natural-language-driven edits. It detects layout, components, and styles visually, generates clean code, and applies iterative visual modifications through simple user instructions.
+
+
+**This Hugging Face repository hosts the `GLM-4.6V` model, part of the `GLM-V` series.**
+
+## Usage
+
+### Environment Installation
+
+For `SGLang`:
+
+```bash
+pip install sglang>=0.5.6.post1
+pip install nvidia-cudnn-cu12==9.16.0.29
+sudo apt update
+sudo apt install ffmpeg
+```
+
+For `vLLM`:
+
+```bash
+pip install vllm>=0.12.0
+pip install transformers>=5.0.0rc0
+```
+
+### Quick Start with Transformers
+
+```python
+from transformers import AutoProcessor, Glm4vMoeForConditionalGeneration
+import torch
+
+MODEL_PATH = "zai-org/GLM-4.6V"
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "url": "https://upload.wikimedia.org/wikipedia/commons/f/fa/Grayscale_8bits_palette_sample_image.png"
+            },
+            {
+                "type": "text",
+                "text": "describe this image"
+            }
+        ],
+    }
+]
+processor = AutoProcessor.from_pretrained(MODEL_PATH)
+model = Glm4vMoeForConditionalGeneration.from_pretrained(
+    pretrained_model_name_or_path=MODEL_PATH,
+    torch_dtype="auto",
+    device_map="auto",
+)
+inputs = processor.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_dict=True,
+    return_tensors="pt"
+).to(model.device)
+inputs.pop("token_type_ids", None)
+generated_ids = model.generate(**inputs, max_new_tokens=8192)
+output_text = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
+print(output_text)
+```
+
+## Evaluation Settings
+
+We primarily use vLLM as the backend for model inference. For faster and more reliable performance on video tasks, we employ SGLang. To reproduce our leaderboard results, we recommend the following decoding parameters:
+
++	top_p: 0.6
++	top_k: 2
++	temperature: 0.8
++	repetition_penalty: 1.1
++	max_generate_tokens: 16K
+
+For more usage details, please refer to Our [Github](https://github.com/zai-org/GLM-V).
+
+## Fixed and Remaining Issues
+
+Since the open-sourcing of GLM-4.1V, we have received extensive feedback from the community and are well aware that the model still has many shortcomings. In subsequent iterations, we attempted to address several common issues — such as repetitive thinking outputs and formatting errors — which have been mitigated to some extent in this new version.
+
+However, the model still has several limitations and issues that we will fix as soon as possible:
+
+1. Pure text QA capabilities still have significant room for improvement. In this development cycle, our primary focus was on visual multimodal scenarios, and we will enhance pure text abilities in upcoming updates.
+2. The model may still overthink or even repeat itself in certain cases, especially when dealing with complex prompts.
+3. In some situations, the model may restate the answer again at the end.
+4. There remain certain perception limitations, such as counting accuracy and identifying specific individuals, which still require improvement.
+
+Thank you for your patience and understanding. We also welcome feedback and suggestions in the issue section — we will respond and improve as much as we can!
+
+## Citation
+
+If you use this model, please cite the following paper:
+
+```bibtex
+@misc{vteam2025glm45vglm41vthinkingversatilemultimodal,
+      title={GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning}, 
+      author={V Team and Wenyi Hong and Wenmeng Yu and Xiaotao Gu and Guo Wang and Guobing Gan and Haomiao Tang and Jiale Cheng and Ji Qi and Junhui Ji and Lihang Pan and Shuaiqi Duan and Weihan Wang and Yan Wang and Yean Cheng and Zehai He and Zhe Su and Zhen Yang and Ziyang Pan and Aohan Zeng and Baoxu Wang and Bin Chen and Boyan Shi and Changyu Pang and Chenhui Zhang and Da Yin and Fan Yang and Guoqing Chen and Jiazheng Xu and Jiale Zhu and Jiali Chen and Jing Chen and Jinhao Chen and Jinghao Lin and Jinjiang Wang and Junjie Chen and Leqi Lei and Letian Gong and Leyi Pan and Mingdao Liu and Mingde Xu and Mingzhi Zhang and Qinkai Zheng and Sheng Yang and Shi Zhong and Shiyu Huang and Shuyuan Zhao and Siyan Xue and Shangqin Tu and Shengbiao Meng and Tianshu Zhang and Tianwei Luo and Tianxiang Hao and Tianyu Tong and Wenkai Li and Wei Jia and Xiao Liu and Xiaohan Zhang and Xin Lyu and Xinyue Fan and Xuancheng Huang and Yanling Wang and Yadong Xue and Yanfeng Wang and Yanzi Wang and Yifan An and Yifan Du and Yiming Shi and Yiheng Huang and Yilin Niu and Yuan Wang and Yuanchang Yue and Yuchen Li and Yutao Zhang and Yuting Wang and Yu Wang and Yuxuan Zhang and Zhao Xue and Zhenyu Hou and Zhengxiao Du and Zihan Wang and Peng Zhang and Debing Liu and Bin Xu and Juanzi Li and Minlie Huang and Yuxiao Dong and Jie Tang},
+      year={2025},
+      eprint={2507.01006},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2507.01006}, 
+}
+```
\ No newline at end of file
diff --git a/chat_template.jinja b/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..384e24338cff1ccbd2907e9ee9dcc753a7e0552d
--- /dev/null
+++ b/chat_template.jinja
@@ -0,0 +1,143 @@
+{# Unsloth template fixes #}
+[gMASK]<sop>
+{%- if tools -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{{ tool | tojson|string }}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}
+<arg_key>{arg-key-1}</arg_key>
+<arg_value>{arg-value-1}</arg_value>
+<arg_key>{arg-key-2}</arg_key>
+<arg_value>{arg-value-2}</arg_value>
+...
+</tool_call>{%- endif -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is mapping and (item.type == 'image' or 'image' in item) -%}
+                <|begin_of_image|><|image|><|end_of_image|>
+            {%- elif item is mapping and (item.type == 'video' or 'video' in item) -%}
+                <|begin_of_video|><|video|><|end_of_video|>
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) %}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {% set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{% for m in messages %}
+{%- if m.role == 'user' -%}<|user|>
+{% if m.content is string %}
+{{ m.content }}
+{%- else %}
+{%- for item in m.content %}
+{% if item.type == 'video' or 'video' in item %}
+<|begin_of_video|><|video|><|end_of_video|>{% elif item.type == 'image' or 'image' in item %}
+<|begin_of_image|><|image|><|end_of_image|>{% elif item.type == 'text' %}
+{{ item.text }}
+{%- endif %}
+{%- endfor %}
+{%- endif %}
+{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set reasoning_content = '' %}
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- else %}
+    {%- if '</think>' in content %}
+        {%- set reasoning_content = ((content.split('</think>')|first).rstrip('\n').split('<think>')|last).lstrip('\n') %}
+        {%- set content = (content.split('</think>')|last).lstrip('\n') %}
+    {%- endif %}
+{%- endif %}
+{%- if loop.index0 > ns.last_user_index and reasoning_content -%}
+{{ '\n<think>' + reasoning_content.strip() +  '</think>'}}
+{%- else -%}
+{{ '\n<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ '\n' + content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{ '\n<tool_call>' + tc.name }}
+{% set _args = tc.arguments %}{% if _args is mapping %}
+{% for k, v in _args|items %}
+<arg_key>{{ k }}</arg_key>
+<arg_value>{{ v | tojson|string if v is not string else v }}</arg_value>
+{% endfor %}{%- endif %}
+</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if m.content is string -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' }}
+{%- endif %}
+{{- '\n<tool_response>\n' }}
+{{- m.content }}
+{{- '\n</tool_response>' }}
+{% elif m.content is iterable and m.content is not mapping %}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+{{- '<|observation|>' }}
+{%- endif %}
+{{- '\n<tool_response>\n' }}
+{%- for tr in m.content -%}
+  {%- if tr is mapping and tr.type is defined -%}
+    {%- set t = tr.type | lower -%}
+    {%- if t == 'text' and tr.text is defined -%}
+{{ tr.text }}
+    {%- elif t in ['image', 'image_url'] -%}
+<|begin_of_image|><|image|><|end_of_image|>
+    {%- elif t in ['video', 'video_url'] -%}
+<|begin_of_video|><|video|><|end_of_video|>
+    {%- else -%}
+{{ tr | tojson|string }}
+    {%- endif -%}
+  {%- else -%}
+{{ tr.output if tr.output is defined else tr }}
+  {%- endif -%}
+{%- endfor -%}
+{{- '\n</tool_response>' }}
+{%- else -%}
+<|observation|>{% for tr in m.content %}
+
+<tool_response>
+{{ tr.output if tr.output is defined else tr }}
+</tool_response>{% endfor -%}
+{% endif -%}
+{# ====== 逻辑结束 ====== #}
+{%- elif m.role == 'system' -%}
+<|system|>
+{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+<|assistant|>
+{{'<think></think>\n' if (enable_thinking is defined and not enable_thinking) else ''}}
+{%- endif -%}
+{# Copyright 2025-present Unsloth. Apache 2.0 License. #}
\ No newline at end of file
diff --git a/config.json b/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..a8f7a3638bd7fac88cc86474dd78b66ee9772f15
--- /dev/null
+++ b/config.json
@@ -0,0 +1,84 @@
+{
+  "architectures": [
+    "Glm4vMoeForConditionalGeneration"
+  ],
+  "image_end_token_id": 151340,
+  "image_start_token_id": 151339,
+  "image_token_id": 151363,
+  "model_type": "glm4v_moe",
+  "pad_token_id": 151330,
+  "text_config": {
+    "attention_bias": true,
+    "attention_dropout": 0.0,
+    "torch_dtype": "bfloat16",
+    "eos_token_id": [
+      151329,
+      151336,
+      151338
+    ],
+    "first_k_dense_replace": 1,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "initializer_range": 0.02,
+    "intermediate_size": 10944,
+    "max_position_embeddings": 131072,
+    "model_type": "Glm4vMoe_text",
+    "moe_intermediate_size": 1408,
+    "n_group": 1,
+    "n_routed_experts": 128,
+    "n_shared_experts": 1,
+    "norm_topk_prob": true,
+    "num_attention_heads": 96,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 46,
+    "num_key_value_heads": 8,
+    "num_nextn_predict_layers": 0,
+    "pad_token_id": 151329,
+    "partial_rotary_factor": 0.5,
+    "qk_layernorm": false,
+    "rms_norm_eps": 1e-05,
+    "rope_parameters": {
+      "mrope_section": [
+        8,
+        12,
+        12
+      ],
+      "partial_rotary_factor": 0.5,
+      "rope_theta": 500000,
+      "rope_type": "default"
+    },
+    "rope_scaling": null,
+    "rope_theta": 10000.0,
+    "routed_scaling_factor": 1.0,
+    "topk_group": 1,
+    "use_cache": true,
+    "use_qk_norm": false,
+    "vocab_size": 151552
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.3",
+  "unsloth_fixed": true,
+  "video_end_token_id": 151342,
+  "video_start_token_id": 151341,
+  "video_token_id": 151364,
+  "vision_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "depth": 24,
+    "hidden_act": "silu",
+    "hidden_dropout_prob": 0.0,
+    "hidden_size": 1536,
+    "image_size": 336,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 10944,
+    "model_type": "glm4v_moe",
+    "num_heads": 12,
+    "out_hidden_size": 4096,
+    "patch_size": 14,
+    "rms_norm_eps": 1e-05,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  }
+}
\ No newline at end of file
diff --git a/generation_config.json b/generation_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..00e2636f2d875aca6c6f12dfa3d23f0a364d337f
--- /dev/null
+++ b/generation_config.json
@@ -0,0 +1,14 @@
+{
+  "_from_model_config": true,
+  "do_sample": true,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338
+  ],
+  "pad_token_id": 151329,
+  "top_p": 0.6,
+  "temperature": 0.8,
+  "top_k": 2,
+  "transformers_version": "5.0.0rc0"
+}
diff --git a/model-00001-of-00041.safetensors b/model-00001-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3859271b9a3e9c5a707c57e6e8a3676d6edb6c79
--- /dev/null
+++ b/model-00001-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38efd9dc38087f7fe8792fbe7f6a0e583f13ce62b8e6ff80d38e497134087e8d
+size 5361983984
diff --git a/model-00002-of-00041.safetensors b/model-00002-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2ef9701fbb5dae7bc1069adf7e80696c284c6d15
--- /dev/null
+++ b/model-00002-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b4a332842fed459e2708a8bb5d068780ab211c91e659feac78ce3cf0b92a463
+size 5363575304
diff --git a/model-00003-of-00041.safetensors b/model-00003-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3e67e9c43a73bf3ae9a503ff96348db69e59b97f
--- /dev/null
+++ b/model-00003-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb8b76a64ffc8ff6a8dc0ecb568865d5d19c908b052fc168a2cf9cf42702f673
+size 5363619584
diff --git a/model-00004-of-00041.safetensors b/model-00004-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..7ae22f6c05ef5b4525996b06f3abb1cb70640cc1
--- /dev/null
+++ b/model-00004-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a086814c1c5ba9d2f7c46b431048275f2333615b9d13a43794e60e6d28e0e521
+size 5363575208
diff --git a/model-00005-of-00041.safetensors b/model-00005-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8631e1efde7a532929b31d6edd116c277b2cdc1b
--- /dev/null
+++ b/model-00005-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:387ae12cb4d537d6458eeacd63f889bfd6088ad7b716a27e1c2bdac95529aebf
+size 5363575208
diff --git a/model-00006-of-00041.safetensors b/model-00006-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..470216e16e528e94faee5861b178fd1360b8e782
--- /dev/null
+++ b/model-00006-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5673a83f72ff619fa58334e1a74e32ec2948f2b5cc563f0394fe0b8eac55958
+size 5363575216
diff --git a/model-00007-of-00041.safetensors b/model-00007-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..20e55975276cf892c654d372b7c9332d3d2a6863
--- /dev/null
+++ b/model-00007-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:752562ac1de0c74b46936c094477b30e3298861f1ffd8d93b1992eada555a804
+size 5363575264
diff --git a/model-00008-of-00041.safetensors b/model-00008-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0dcede6804c1ac01ed35dc34d715fc6c6d7b9008
--- /dev/null
+++ b/model-00008-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b46a85b019fac2d48db5f7b4081b59cf707b1d5c1f85814fd0b876ccc7a7639
+size 5363575264
diff --git a/model-00009-of-00041.safetensors b/model-00009-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..fd19c3f2ff04874882987a065e965b6cc10bbfd9
--- /dev/null
+++ b/model-00009-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90421e485b9471b4daa340631504ac8d752305cc7b2e9c17fc68821edfcec8c0
+size 5363575688
diff --git a/model-00010-of-00041.safetensors b/model-00010-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..511d7af8ca5793c1b69edcfa527f7fed41fd8b46
--- /dev/null
+++ b/model-00010-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ddbca9c51e257de612221ef0b8be3b72dc0b733cf5bdb44e8e7c495eee4f8e3
+size 5363620008
diff --git a/model-00011-of-00041.safetensors b/model-00011-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ee8d6e492791a55d8c43180f67693db5058b34bb
--- /dev/null
+++ b/model-00011-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a776706ccfffca79c6d66017b30aa6485b094220e335ecfb5bf0bf998789141d
+size 5363575672
diff --git a/model-00012-of-00041.safetensors b/model-00012-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..567e2f929b59693bd7b3a7cbdfda6b743d1ae3ac
--- /dev/null
+++ b/model-00012-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a94644ff5dd114cf5db9509b2fe6bb1181e86024966da49e06df78f3def6efc1
+size 5363575664
diff --git a/model-00013-of-00041.safetensors b/model-00013-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..74f16432187fa74adfeff4c9f7c1b0f795b66690
--- /dev/null
+++ b/model-00013-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d0b2473ee2057103fd6b79fab2d81c310842689ce41bd30314774e157f0a4a2
+size 5363575680
diff --git a/model-00014-of-00041.safetensors b/model-00014-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..7d04d57beab842eeb0d63989d1649b09202a47b6
--- /dev/null
+++ b/model-00014-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efe6ecdb2ed438f81346f6ef657dcdda1c7d13bbd335127ddd8b43e1e992cb5d
+size 5363575712
diff --git a/model-00015-of-00041.safetensors b/model-00015-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..974e9c6d467d10d261da5683c04ceaaaef7d9e95
--- /dev/null
+++ b/model-00015-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:150baaa2f3293d8f17b2829e5e34ff86e26bc765cc59821ba8d67a41a534c807
+size 5363575728
diff --git a/model-00016-of-00041.safetensors b/model-00016-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..81dc0d85958730f9504abd7425705258492a5d0b
--- /dev/null
+++ b/model-00016-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:411f03b4f10deda00999f133a5f61ae66e0daed8bcda3be1e9eb4a0288dd3764
+size 5318486568
diff --git a/model-00017-of-00041.safetensors b/model-00017-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e53cff23fcce6c72ca324f3c034a064436582baf
--- /dev/null
+++ b/model-00017-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36682d1b038fb94a5e812c144ea0b58aa7ccc708ccf201fbc854a2c647454573
+size 5362571320
diff --git a/model-00018-of-00041.safetensors b/model-00018-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d61978009602bfba5d9a48ec25640de839a5ef7b
--- /dev/null
+++ b/model-00018-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16364dccc11dfd59aa57e559128256e638b1583e428bb9193f65b020adfd9d69
+size 5363575664
diff --git a/model-00019-of-00041.safetensors b/model-00019-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..cbeac7e066e0d98bb6754afcee9852fc96ef0097
--- /dev/null
+++ b/model-00019-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60c7347f5716e2c09709431b6937012dc5e92d1f3d586e557f6516605333b688
+size 5363575672
diff --git a/model-00020-of-00041.safetensors b/model-00020-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..292bab62006836acd9e44dc6121a7ef5b6c897dd
--- /dev/null
+++ b/model-00020-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:893764ce042b2e2f8f3ab6bbb8789c2681b0d0106f7f9ac96a922a79bdf9c7fc
+size 5363575680
diff --git a/model-00021-of-00041.safetensors b/model-00021-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b8073992c5be07aae9923eb32853473845bfaece
--- /dev/null
+++ b/model-00021-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:183fdf7aa13f9515ffdd54ca3ff0bea02817ae785cb34a8f10952f515f1e33a3
+size 5363575720
diff --git a/model-00022-of-00041.safetensors b/model-00022-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..acb5d5561b1fd848f45f4b30c1beee1694ccc65d
--- /dev/null
+++ b/model-00022-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc65d473f840f71b34ed91129a9814ec8823e2589fc6e0111dca021e830b0026
+size 5363575728
diff --git a/model-00023-of-00041.safetensors b/model-00023-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..85f92ff6b38f15a137bd96699329a0cbeb0a92aa
--- /dev/null
+++ b/model-00023-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7922973bc3559e3ba29d80e57f0a5ee0af2efa5c156e329c5ac014695aede7a
+size 5318494904
diff --git a/model-00024-of-00041.safetensors b/model-00024-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..76ebf398bebba5af18d2d7934bbccc55e15d6f5d
--- /dev/null
+++ b/model-00024-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35f0a81cb00bc4b7fd1f844d187a81d9314057d2cf25d3e6d7cc0daa3d560a25
+size 5362562976
diff --git a/model-00025-of-00041.safetensors b/model-00025-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..58ba2184dbf3dc1ad1f9174f7f00b6113917fe07
--- /dev/null
+++ b/model-00025-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cae885012e402a368377d14df78737ff885a48f3100f9772e8d24ac54d4d62b0
+size 5363575672
diff --git a/model-00026-of-00041.safetensors b/model-00026-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b8faf84ba37d375b12a45c4e5706874be418552b
--- /dev/null
+++ b/model-00026-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfd5a81a19ee4b6ff4d4d95adcf188727bf9d598cd15d856c29a6bbcdc8cc7f
+size 5363575664
diff --git a/model-00027-of-00041.safetensors b/model-00027-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a4fb722817cf2caa2e6ffb8917becf4b279cd240
--- /dev/null
+++ b/model-00027-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:251d334b8152a4b211d2c90ed1d098ce12ad0c7f184f3ecc0554bdf312ae3ab8
+size 5363575688
diff --git a/model-00028-of-00041.safetensors b/model-00028-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..fcaae07776d8ea3073c93321f563870f80d792bf
--- /dev/null
+++ b/model-00028-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:610a71efa3bbee04f23a0f58f6637f5ea55ac78a5d07bcb34000facab4d4e661
+size 5363575712
diff --git a/model-00029-of-00041.safetensors b/model-00029-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2e0590551b9b56dca43e868bfd613e1a5505ad0b
--- /dev/null
+++ b/model-00029-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fbcf540b11d006c7520edfdbdc28652c7dfdff1b9376b3e7001c4258bac492d
+size 5363575728
diff --git a/model-00030-of-00041.safetensors b/model-00030-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8b0d867ec6376f0ca56e51c1ddf7dc22aab26d15
--- /dev/null
+++ b/model-00030-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e797634303d7bfa09f658f00201b6a4965f899054d94ec96e4dad364a234d14
+size 5360428912
diff --git a/model-00031-of-00041.safetensors b/model-00031-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..20ca70c8d1523748620774f1618fc56405b2f073
--- /dev/null
+++ b/model-00031-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16f83c7b0cb5955bab3911ae7ded75214ddf91f0c00395140b6ca421f7179a32
+size 5366766840
diff --git a/model-00032-of-00041.safetensors b/model-00032-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..663f34d98290cb139514a7fc48244d8a5604d3f7
--- /dev/null
+++ b/model-00032-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d1db849fed8ffb7172ef96bb4cabf0ffe9fc8913ab34c5f7e8c1d2a48eb5e4d
+size 5363575664
diff --git a/model-00033-of-00041.safetensors b/model-00033-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d76797acaa3440457c9b8229254b540bc25991fd
--- /dev/null
+++ b/model-00033-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72db0851df1f68190dc42634e4118ccf225d2152571e6a19b80b0c9493a50d11
+size 5363575672
diff --git a/model-00034-of-00041.safetensors b/model-00034-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a5a15551739a70962b14cd96d0ee94e00bc116aa
--- /dev/null
+++ b/model-00034-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26c669f9799d73f64a958ce73dab53188350638d0e97f2030f0a3a4e3a7ec28a
+size 5363575688
diff --git a/model-00035-of-00041.safetensors b/model-00035-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..081474c7b3dd524af29386f3c4758904f3b63984
--- /dev/null
+++ b/model-00035-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec3347c40247896ffbee51922115a18373b4085279b984b1fe5f8c1e3fcf1fd2
+size 5363575720
diff --git a/model-00036-of-00041.safetensors b/model-00036-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e4df473b8cac5ada78f4907903aba0f1cb0812db
--- /dev/null
+++ b/model-00036-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f28fdf954eec0fb03f40133631ec308c3c4aa10494c1daa8899bd66951e5143
+size 5363575728
diff --git a/model-00037-of-00041.safetensors b/model-00037-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e17807a2a3b9f80616469e29b142816711bdf6da
--- /dev/null
+++ b/model-00037-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70b8730b4fde3af5a9ee31618480b2a8dbb5394c002950f4d2f4794eb3fc936d
+size 5320583352
diff --git a/model-00038-of-00041.safetensors b/model-00038-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1b599577db144fb04d6d4b8e4c4d8789d2f99851
--- /dev/null
+++ b/model-00038-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:941fadb4f1eb630f85babe2e5747f225c3648b0be9b15668563368781b152aa0
+size 5360474496
diff --git a/model-00039-of-00041.safetensors b/model-00039-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bc5d0f7bcdd3db2a964d53478b93b70084a859fa
--- /dev/null
+++ b/model-00039-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e83e3c2198edd7fbf158c6bab988987a3450777a2fb1941f847abddf5dfdc19
+size 5363575672
diff --git a/model-00040-of-00041.safetensors b/model-00040-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c22455b0864dc35b204e9b6929d2da7c4ba62f25
--- /dev/null
+++ b/model-00040-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a0dcdd1d8f5e5214e45b4e0b3573a43d914cf68539e5426f499ff3673b4b351
+size 5356227208
diff --git a/model-00041-of-00041.safetensors b/model-00041-of-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e5a8e12cad4f56f57246c5571417179dbf0d6ce0
--- /dev/null
+++ b/model-00041-of-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66042701544f10ea793f2e326343bdf9cc5d55f20cc725773dac815d5d0efc8b
+size 1028463544
diff --git a/model.safetensors.index.json b/model.safetensors.index.json
new file mode 100644
index 0000000000000000000000000000000000000000..67dfe8031a72c87ec5d3ffecf7f4fb1579423330
--- /dev/null
+++ b/model.safetensors.index.json
@@ -0,0 +1,18113 @@
+{
+  "metadata": {
+    "total_size": 107710933120
+  },
+  "weight_map": {
+    "model.language_model.embed_tokens.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.q_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.k_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.v_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.o_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.mlp.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.mlp.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.mlp.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.q_proj.bias": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.k_proj.bias": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.self_attn.v_proj.bias": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.input_layernorm.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.0.post_attention_layernorm.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.0.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.0.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.1.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.1.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.2.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.2.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.3.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.3.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.4.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.4.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.5.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.5.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.6.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.6.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.7.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.7.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.8.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.8.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.9.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.9.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.10.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.10.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.11.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.11.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.12.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.12.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.13.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.13.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.14.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.14.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.15.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.15.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.16.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.16.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.17.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.17.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.18.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.18.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.19.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.19.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.20.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.20.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.21.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.21.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.22.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.22.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.23.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.23.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.24.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.24.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.25.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.25.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.26.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.26.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.27.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.27.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.28.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.28.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.29.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.29.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.30.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.30.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.31.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.31.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.32.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.32.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.33.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.33.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.34.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.34.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.35.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.35.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.36.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.36.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.37.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.37.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.38.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.38.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.39.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.39.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.40.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.40.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.41.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.41.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.42.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.42.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.43.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.43.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.44.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.44.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.45.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.45.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.46.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.46.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.47.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.47.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.48.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.48.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.49.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.49.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.50.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.50.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.51.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.51.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.52.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.52.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.53.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.53.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.54.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.54.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.55.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.55.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.56.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.56.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.57.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.57.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.58.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.58.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.59.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.59.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.60.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.60.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.61.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.61.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.62.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.62.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.63.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.63.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.64.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.64.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.65.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.65.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.66.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.66.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.67.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.67.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.68.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.68.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.69.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.69.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.70.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.70.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.71.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.71.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.72.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.72.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.73.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.73.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.74.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.74.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.75.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.75.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.76.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.76.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.77.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.77.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.78.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.78.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.79.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.79.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.80.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.80.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.81.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.81.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.82.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.82.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.83.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.83.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.84.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.84.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.85.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.85.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.86.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.86.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.87.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.87.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.88.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.88.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.89.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.89.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.90.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.90.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.91.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.91.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.92.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.92.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.93.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.93.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.94.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.94.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.95.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.95.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.96.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.96.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.97.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.97.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.98.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.98.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.99.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.99.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.100.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.100.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.101.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.101.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.102.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.102.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.103.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.103.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.104.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.104.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.105.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.105.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.106.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.106.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.107.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.107.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.108.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.108.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.109.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.109.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.110.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.110.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.111.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.111.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.112.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.112.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.113.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.113.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.114.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.114.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.115.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.115.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.116.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.116.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.117.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.117.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.118.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.118.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.119.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.119.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.120.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.120.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.121.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.121.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.122.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.122.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.123.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.123.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.124.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.124.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.125.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.125.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.126.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.126.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.127.gate_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.127.up_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.0.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.1.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.2.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.3.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.4.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.5.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.6.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.7.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.8.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.9.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.10.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.11.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.12.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.13.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.14.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.15.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.16.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.17.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.18.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.19.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.20.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.21.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.22.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.23.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.24.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.25.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.26.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.27.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.28.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.29.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.30.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.31.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.32.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.33.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.34.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.35.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.36.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.37.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.38.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.39.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.40.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.41.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.42.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.43.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.44.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.45.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.46.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.47.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.48.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.49.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.50.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.51.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.52.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.53.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.54.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.55.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.56.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.57.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.58.down_proj.weight": "model-00001-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.59.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.60.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.61.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.62.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.63.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.64.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.65.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.66.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.67.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.68.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.69.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.70.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.71.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.72.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.73.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.74.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.75.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.76.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.77.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.78.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.79.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.80.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.81.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.82.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.83.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.84.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.85.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.86.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.87.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.88.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.89.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.90.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.91.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.92.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.93.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.94.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.95.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.96.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.97.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.98.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.99.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.100.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.101.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.102.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.103.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.104.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.105.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.106.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.107.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.108.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.109.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.110.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.111.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.112.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.113.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.114.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.115.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.116.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.117.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.118.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.119.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.120.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.121.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.122.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.123.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.124.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.125.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.126.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.experts.127.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.q_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.k_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.v_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.shared_experts.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.shared_experts.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.shared_experts.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.gate.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.o_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.input_layernorm.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.q_proj.bias": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.k_proj.bias": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.self_attn.v_proj.bias": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.post_attention_layernorm.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.1.mlp.gate.e_score_correction_bias": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.0.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.0.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.1.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.1.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.2.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.2.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.3.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.3.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.4.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.4.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.5.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.5.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.6.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.6.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.7.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.7.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.8.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.8.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.9.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.9.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.10.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.10.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.11.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.11.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.12.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.12.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.13.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.13.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.14.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.14.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.15.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.15.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.16.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.16.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.17.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.17.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.18.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.18.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.19.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.19.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.20.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.20.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.21.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.21.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.22.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.22.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.23.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.23.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.24.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.24.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.25.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.25.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.26.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.26.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.27.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.27.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.28.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.28.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.29.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.29.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.30.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.30.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.31.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.31.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.32.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.32.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.33.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.33.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.34.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.34.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.35.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.35.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.36.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.36.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.37.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.37.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.38.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.38.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.39.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.39.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.40.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.40.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.41.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.41.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.42.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.42.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.43.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.43.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.44.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.44.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.45.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.45.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.46.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.46.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.47.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.47.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.48.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.48.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.49.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.49.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.50.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.50.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.51.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.51.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.52.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.52.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.53.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.53.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.54.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.54.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.55.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.55.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.56.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.56.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.57.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.57.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.58.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.58.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.59.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.59.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.60.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.60.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.61.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.61.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.62.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.62.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.63.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.63.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.64.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.64.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.65.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.65.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.66.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.66.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.67.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.67.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.68.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.68.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.69.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.69.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.70.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.70.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.71.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.71.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.72.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.72.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.73.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.73.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.74.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.74.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.75.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.75.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.76.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.76.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.77.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.77.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.78.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.78.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.79.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.79.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.80.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.80.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.81.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.81.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.82.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.82.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.83.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.83.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.84.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.84.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.85.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.85.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.86.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.86.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.87.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.87.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.88.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.88.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.89.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.89.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.90.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.90.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.91.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.91.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.92.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.92.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.93.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.93.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.94.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.94.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.95.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.95.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.96.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.96.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.97.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.97.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.98.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.98.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.99.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.99.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.100.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.100.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.101.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.101.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.102.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.102.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.103.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.103.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.104.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.104.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.105.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.105.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.106.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.106.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.107.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.107.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.108.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.108.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.109.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.109.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.110.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.110.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.111.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.111.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.112.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.112.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.113.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.113.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.114.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.114.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.115.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.115.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.116.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.116.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.117.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.117.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.118.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.118.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.119.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.119.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.120.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.120.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.121.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.121.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.122.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.122.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.123.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.123.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.124.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.124.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.125.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.125.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.126.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.126.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.127.gate_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.127.up_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.0.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.1.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.2.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.3.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.4.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.5.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.6.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.7.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.8.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.9.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.10.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.11.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.12.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.13.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.14.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.15.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.16.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.17.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.18.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.19.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.20.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.21.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.22.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.23.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.24.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.25.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.26.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.27.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.28.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.29.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.30.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.31.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.32.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.33.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.34.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.35.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.36.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.37.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.38.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.39.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.40.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.41.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.42.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.43.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.44.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.45.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.46.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.47.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.48.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.49.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.50.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.51.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.52.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.53.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.54.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.55.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.56.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.57.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.58.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.59.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.60.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.61.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.62.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.63.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.64.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.65.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.66.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.67.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.68.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.69.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.70.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.71.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.72.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.73.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.74.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.75.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.76.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.77.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.78.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.79.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.80.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.81.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.82.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.83.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.84.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.85.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.86.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.87.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.88.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.89.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.90.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.91.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.92.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.93.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.94.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.95.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.96.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.97.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.98.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.99.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.100.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.101.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.102.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.103.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.104.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.105.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.106.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.107.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.108.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.109.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.110.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.111.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.112.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.113.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.114.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.115.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.116.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.117.down_proj.weight": "model-00002-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.118.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.119.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.120.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.121.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.122.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.123.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.124.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.125.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.126.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.experts.127.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.shared_experts.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.shared_experts.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.gate.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.q_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.k_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.v_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.q_proj.bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.k_proj.bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.v_proj.bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.shared_experts.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.self_attn.o_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.post_attention_layernorm.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.input_layernorm.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.2.mlp.gate.e_score_correction_bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.0.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.0.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.1.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.1.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.2.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.2.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.3.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.3.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.4.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.4.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.5.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.5.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.6.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.6.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.7.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.7.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.8.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.8.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.9.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.9.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.10.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.10.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.11.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.11.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.12.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.12.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.13.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.13.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.14.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.14.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.15.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.15.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.16.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.16.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.17.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.17.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.18.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.18.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.19.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.19.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.20.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.20.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.21.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.21.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.22.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.22.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.23.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.23.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.24.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.24.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.25.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.25.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.26.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.26.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.27.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.27.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.28.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.28.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.29.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.29.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.30.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.30.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.31.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.31.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.32.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.32.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.33.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.33.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.34.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.34.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.35.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.35.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.36.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.36.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.37.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.37.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.38.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.38.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.39.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.39.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.40.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.40.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.41.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.41.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.42.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.42.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.43.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.43.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.44.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.44.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.45.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.45.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.46.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.46.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.47.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.47.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.48.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.48.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.49.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.49.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.50.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.50.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.51.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.51.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.52.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.52.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.53.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.53.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.54.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.54.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.55.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.55.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.56.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.56.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.57.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.57.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.58.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.58.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.59.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.59.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.60.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.60.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.61.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.61.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.62.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.62.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.63.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.63.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.64.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.64.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.65.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.65.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.66.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.66.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.67.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.67.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.68.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.68.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.69.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.69.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.70.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.70.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.71.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.71.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.72.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.72.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.73.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.73.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.74.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.74.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.75.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.75.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.76.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.76.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.77.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.77.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.78.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.78.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.79.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.79.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.80.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.80.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.81.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.81.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.82.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.82.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.83.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.83.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.84.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.84.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.85.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.85.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.86.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.86.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.87.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.87.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.88.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.88.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.89.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.89.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.90.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.90.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.91.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.91.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.92.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.92.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.93.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.93.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.94.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.94.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.95.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.95.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.96.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.96.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.97.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.97.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.98.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.98.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.99.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.99.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.100.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.100.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.101.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.101.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.102.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.102.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.103.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.103.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.104.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.104.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.105.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.105.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.106.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.106.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.107.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.107.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.108.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.108.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.109.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.109.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.110.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.110.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.111.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.111.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.112.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.112.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.113.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.113.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.114.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.114.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.115.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.115.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.116.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.116.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.117.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.117.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.118.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.118.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.119.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.119.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.120.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.120.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.121.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.121.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.122.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.122.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.123.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.123.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.124.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.124.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.125.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.125.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.126.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.126.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.127.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.127.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.0.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.1.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.2.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.3.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.4.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.5.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.6.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.7.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.8.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.9.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.10.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.11.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.12.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.13.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.14.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.15.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.16.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.17.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.18.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.19.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.20.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.21.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.22.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.23.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.24.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.25.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.26.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.27.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.28.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.29.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.30.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.31.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.32.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.33.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.34.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.35.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.36.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.37.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.38.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.39.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.40.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.41.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.42.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.43.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.44.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.45.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.46.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.47.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.48.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.49.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.50.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.51.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.52.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.53.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.54.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.55.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.56.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.57.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.58.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.59.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.60.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.61.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.62.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.63.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.64.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.65.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.66.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.67.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.68.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.69.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.70.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.71.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.72.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.73.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.74.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.75.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.76.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.77.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.78.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.79.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.80.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.81.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.82.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.83.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.84.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.85.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.86.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.87.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.88.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.89.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.90.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.91.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.92.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.93.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.94.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.95.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.96.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.97.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.98.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.99.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.100.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.101.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.102.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.103.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.104.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.105.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.106.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.107.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.108.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.109.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.110.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.111.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.112.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.113.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.114.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.115.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.116.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.117.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.118.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.119.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.120.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.121.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.122.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.123.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.124.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.125.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.126.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.experts.127.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.gate.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.shared_experts.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.shared_experts.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.shared_experts.down_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.q_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.k_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.v_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.mlp.gate.e_score_correction_bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.o_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.post_attention_layernorm.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.input_layernorm.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.q_proj.bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.k_proj.bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.3.self_attn.v_proj.bias": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.0.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.0.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.1.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.1.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.2.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.2.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.3.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.3.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.4.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.4.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.5.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.5.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.6.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.6.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.7.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.7.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.8.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.8.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.9.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.9.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.10.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.10.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.11.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.11.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.12.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.12.up_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.13.gate_proj.weight": "model-00003-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.13.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.14.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.14.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.15.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.15.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.16.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.16.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.17.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.17.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.18.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.18.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.19.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.19.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.20.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.20.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.21.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.21.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.22.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.22.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.23.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.23.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.24.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.24.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.25.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.25.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.26.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.26.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.27.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.27.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.28.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.28.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.29.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.29.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.30.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.30.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.31.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.31.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.32.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.32.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.33.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.33.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.34.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.34.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.35.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.35.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.36.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.36.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.37.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.37.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.38.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.38.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.39.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.39.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.40.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.40.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.41.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.41.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.42.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.42.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.43.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.43.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.44.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.44.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.45.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.45.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.46.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.46.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.47.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.47.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.48.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.48.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.49.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.49.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.50.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.50.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.51.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.51.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.52.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.52.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.53.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.53.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.54.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.54.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.55.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.55.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.56.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.56.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.57.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.57.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.58.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.58.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.59.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.59.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.60.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.60.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.61.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.61.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.62.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.62.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.63.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.63.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.64.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.64.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.65.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.65.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.66.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.66.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.67.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.67.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.68.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.68.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.69.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.69.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.70.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.70.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.71.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.71.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.72.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.72.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.73.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.73.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.74.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.74.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.75.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.75.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.76.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.76.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.77.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.77.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.78.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.78.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.79.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.79.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.80.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.80.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.81.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.81.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.82.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.82.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.83.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.83.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.84.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.84.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.85.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.85.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.86.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.86.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.87.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.87.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.88.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.88.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.89.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.89.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.90.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.90.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.91.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.91.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.92.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.92.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.93.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.93.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.94.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.94.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.95.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.95.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.96.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.96.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.97.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.97.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.98.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.98.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.99.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.99.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.100.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.100.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.101.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.101.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.102.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.102.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.103.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.103.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.104.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.104.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.105.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.105.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.106.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.106.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.107.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.107.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.108.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.108.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.109.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.109.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.110.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.110.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.111.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.111.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.112.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.112.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.113.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.113.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.114.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.114.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.115.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.115.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.116.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.116.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.117.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.117.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.118.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.118.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.119.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.119.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.120.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.120.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.121.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.121.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.122.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.122.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.123.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.123.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.124.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.124.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.125.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.125.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.126.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.126.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.127.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.127.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.0.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.1.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.2.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.3.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.4.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.5.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.6.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.7.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.8.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.9.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.10.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.11.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.12.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.13.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.14.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.15.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.16.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.17.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.18.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.19.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.20.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.21.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.22.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.23.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.24.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.25.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.26.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.27.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.28.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.29.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.30.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.31.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.32.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.33.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.34.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.35.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.36.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.37.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.38.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.39.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.40.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.41.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.42.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.43.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.44.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.45.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.46.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.47.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.48.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.49.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.50.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.51.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.52.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.53.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.54.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.55.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.56.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.57.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.58.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.59.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.60.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.61.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.62.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.63.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.64.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.65.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.66.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.67.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.68.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.69.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.70.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.71.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.72.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.73.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.74.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.75.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.76.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.77.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.78.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.79.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.80.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.81.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.82.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.83.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.84.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.85.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.86.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.87.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.88.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.89.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.90.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.91.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.92.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.93.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.94.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.95.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.96.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.97.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.98.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.99.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.100.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.101.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.102.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.103.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.104.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.105.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.106.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.107.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.108.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.109.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.110.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.111.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.112.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.113.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.114.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.115.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.116.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.117.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.118.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.119.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.120.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.121.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.122.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.123.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.124.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.125.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.126.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.experts.127.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.q_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.k_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.v_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.o_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.post_attention_layernorm.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.gate.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.shared_experts.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.shared_experts.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.shared_experts.down_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.input_layernorm.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.mlp.gate.e_score_correction_bias": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.q_proj.bias": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.k_proj.bias": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.4.self_attn.v_proj.bias": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.0.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.0.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.1.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.1.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.2.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.2.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.3.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.3.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.4.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.4.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.5.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.5.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.6.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.6.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.7.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.7.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.8.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.8.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.9.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.9.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.10.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.10.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.11.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.11.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.12.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.12.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.13.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.13.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.14.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.14.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.15.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.15.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.16.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.16.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.17.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.17.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.18.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.18.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.19.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.19.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.20.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.20.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.21.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.21.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.22.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.22.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.23.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.23.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.24.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.24.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.25.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.25.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.26.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.26.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.27.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.27.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.28.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.28.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.29.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.29.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.30.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.30.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.31.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.31.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.32.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.32.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.33.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.33.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.34.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.34.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.35.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.35.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.36.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.36.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.37.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.37.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.38.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.38.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.39.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.39.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.40.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.40.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.41.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.41.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.42.gate_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.42.up_proj.weight": "model-00004-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.43.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.43.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.44.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.44.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.45.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.45.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.46.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.46.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.47.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.47.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.48.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.48.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.49.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.49.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.50.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.50.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.51.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.51.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.52.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.52.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.53.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.53.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.54.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.54.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.55.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.55.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.56.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.56.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.57.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.57.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.58.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.58.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.59.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.59.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.60.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.60.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.61.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.61.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.62.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.62.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.63.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.63.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.64.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.64.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.65.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.65.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.66.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.66.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.67.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.67.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.68.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.68.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.69.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.69.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.70.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.70.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.71.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.71.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.72.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.72.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.73.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.73.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.74.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.74.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.75.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.75.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.76.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.76.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.77.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.77.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.78.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.78.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.79.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.79.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.80.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.80.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.81.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.81.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.82.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.82.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.83.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.83.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.84.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.84.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.85.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.85.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.86.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.86.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.87.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.87.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.88.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.88.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.89.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.89.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.90.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.90.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.91.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.91.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.92.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.92.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.93.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.93.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.94.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.94.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.95.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.95.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.96.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.96.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.97.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.97.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.98.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.98.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.99.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.99.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.100.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.100.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.101.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.101.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.102.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.102.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.103.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.103.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.104.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.104.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.105.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.105.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.106.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.106.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.107.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.107.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.108.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.108.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.109.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.109.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.110.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.110.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.111.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.111.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.112.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.112.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.113.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.113.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.114.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.114.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.115.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.115.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.116.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.116.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.117.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.117.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.118.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.118.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.119.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.119.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.120.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.120.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.121.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.121.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.122.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.122.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.123.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.123.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.124.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.124.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.125.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.125.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.126.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.126.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.127.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.127.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.0.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.1.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.2.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.3.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.4.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.5.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.6.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.7.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.8.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.9.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.10.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.11.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.12.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.13.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.14.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.15.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.16.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.17.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.18.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.19.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.20.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.21.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.22.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.23.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.24.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.25.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.26.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.27.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.28.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.29.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.30.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.31.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.32.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.33.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.34.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.35.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.36.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.37.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.38.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.39.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.40.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.41.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.42.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.43.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.44.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.45.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.46.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.47.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.48.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.49.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.50.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.51.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.52.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.53.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.54.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.55.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.56.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.57.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.58.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.59.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.60.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.61.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.62.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.63.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.64.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.65.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.66.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.67.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.68.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.69.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.70.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.71.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.72.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.73.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.74.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.75.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.76.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.77.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.78.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.79.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.80.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.81.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.82.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.83.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.84.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.85.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.86.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.87.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.88.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.89.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.90.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.91.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.92.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.93.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.94.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.95.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.96.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.97.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.98.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.99.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.100.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.101.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.102.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.103.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.104.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.105.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.106.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.107.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.108.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.109.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.110.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.111.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.112.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.113.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.114.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.115.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.116.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.117.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.118.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.119.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.120.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.121.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.122.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.123.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.124.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.125.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.126.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.experts.127.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.shared_experts.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.shared_experts.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.shared_experts.down_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.input_layernorm.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.q_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.k_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.v_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.gate.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.o_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.q_proj.bias": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.k_proj.bias": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.self_attn.v_proj.bias": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.post_attention_layernorm.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.5.mlp.gate.e_score_correction_bias": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.0.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.0.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.1.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.1.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.2.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.2.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.3.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.3.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.4.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.4.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.5.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.5.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.6.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.6.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.7.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.7.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.8.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.8.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.9.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.9.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.10.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.10.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.11.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.11.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.12.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.12.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.13.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.13.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.14.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.14.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.15.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.15.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.16.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.16.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.17.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.17.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.18.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.18.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.19.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.19.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.20.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.20.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.21.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.21.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.22.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.22.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.23.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.23.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.24.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.24.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.25.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.25.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.26.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.26.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.27.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.27.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.28.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.28.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.29.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.29.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.30.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.30.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.31.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.31.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.32.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.32.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.33.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.33.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.34.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.34.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.35.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.35.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.36.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.36.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.37.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.37.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.38.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.38.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.39.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.39.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.40.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.40.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.41.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.41.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.42.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.42.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.43.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.43.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.44.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.44.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.45.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.45.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.46.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.46.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.47.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.47.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.48.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.48.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.49.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.49.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.50.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.50.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.51.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.51.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.52.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.52.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.53.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.53.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.54.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.54.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.55.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.55.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.56.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.56.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.57.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.57.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.58.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.58.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.59.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.59.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.60.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.60.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.61.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.61.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.62.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.62.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.63.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.63.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.64.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.64.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.65.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.65.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.66.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.66.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.67.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.67.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.68.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.68.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.69.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.69.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.70.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.70.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.71.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.71.up_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.72.gate_proj.weight": "model-00005-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.72.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.73.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.73.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.74.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.74.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.75.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.75.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.76.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.76.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.77.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.77.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.78.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.78.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.79.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.79.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.80.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.80.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.81.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.81.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.82.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.82.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.83.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.83.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.84.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.84.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.85.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.85.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.86.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.86.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.87.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.87.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.88.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.88.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.89.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.89.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.90.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.90.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.91.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.91.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.92.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.92.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.93.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.93.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.94.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.94.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.95.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.95.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.96.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.96.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.97.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.97.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.98.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.98.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.99.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.99.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.100.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.100.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.101.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.101.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.102.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.102.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.103.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.103.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.104.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.104.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.105.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.105.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.106.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.106.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.107.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.107.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.108.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.108.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.109.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.109.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.110.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.110.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.111.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.111.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.112.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.112.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.113.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.113.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.114.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.114.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.115.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.115.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.116.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.116.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.117.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.117.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.118.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.118.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.119.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.119.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.120.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.120.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.121.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.121.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.122.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.122.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.123.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.123.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.124.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.124.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.125.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.125.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.126.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.126.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.127.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.127.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.0.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.1.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.2.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.3.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.4.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.5.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.6.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.7.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.8.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.9.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.10.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.11.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.12.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.13.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.14.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.15.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.16.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.17.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.18.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.19.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.20.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.21.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.22.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.23.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.24.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.25.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.26.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.27.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.28.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.29.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.30.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.31.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.32.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.33.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.34.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.35.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.36.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.37.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.38.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.39.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.40.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.41.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.42.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.43.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.44.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.45.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.46.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.47.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.48.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.49.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.50.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.51.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.52.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.53.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.54.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.55.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.56.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.57.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.58.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.59.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.60.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.61.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.62.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.63.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.64.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.65.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.66.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.67.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.68.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.69.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.70.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.71.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.72.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.73.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.74.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.75.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.76.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.77.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.78.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.79.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.80.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.81.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.82.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.83.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.84.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.85.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.86.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.87.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.88.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.89.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.90.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.91.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.92.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.93.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.94.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.95.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.96.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.97.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.98.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.99.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.100.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.101.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.102.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.103.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.104.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.105.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.106.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.107.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.108.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.109.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.110.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.111.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.112.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.113.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.114.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.115.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.116.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.117.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.118.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.119.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.120.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.121.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.122.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.123.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.124.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.125.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.126.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.experts.127.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.gate.e_score_correction_bias": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.q_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.k_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.v_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.gate.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.o_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.shared_experts.down_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.q_proj.bias": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.k_proj.bias": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.self_attn.v_proj.bias": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.post_attention_layernorm.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.shared_experts.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.mlp.shared_experts.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.6.input_layernorm.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.0.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.0.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.1.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.1.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.2.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.2.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.3.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.3.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.4.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.4.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.5.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.5.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.6.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.6.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.7.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.7.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.8.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.8.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.9.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.9.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.10.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.10.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.11.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.11.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.12.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.12.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.13.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.13.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.14.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.14.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.15.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.15.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.16.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.16.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.17.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.17.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.18.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.18.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.19.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.19.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.20.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.20.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.21.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.21.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.22.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.22.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.23.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.23.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.24.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.24.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.25.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.25.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.26.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.26.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.27.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.27.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.28.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.28.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.29.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.29.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.30.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.30.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.31.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.31.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.32.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.32.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.33.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.33.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.34.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.34.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.35.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.35.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.36.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.36.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.37.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.37.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.38.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.38.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.39.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.39.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.40.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.40.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.41.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.41.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.42.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.42.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.43.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.43.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.44.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.44.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.45.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.45.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.46.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.46.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.47.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.47.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.48.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.48.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.49.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.49.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.50.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.50.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.51.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.51.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.52.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.52.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.53.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.53.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.54.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.54.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.55.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.55.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.56.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.56.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.57.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.57.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.58.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.58.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.59.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.59.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.60.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.60.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.61.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.61.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.62.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.62.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.63.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.63.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.64.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.64.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.65.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.65.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.66.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.66.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.67.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.67.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.68.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.68.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.69.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.69.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.70.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.70.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.71.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.71.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.72.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.72.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.73.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.73.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.74.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.74.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.75.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.75.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.76.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.76.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.77.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.77.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.78.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.78.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.79.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.79.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.80.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.80.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.81.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.81.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.82.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.82.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.83.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.83.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.84.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.84.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.85.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.85.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.86.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.86.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.87.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.87.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.88.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.88.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.89.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.89.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.90.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.90.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.91.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.91.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.92.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.92.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.93.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.93.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.94.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.94.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.95.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.95.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.96.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.96.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.97.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.97.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.98.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.98.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.99.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.99.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.100.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.100.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.101.gate_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.101.up_proj.weight": "model-00006-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.102.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.102.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.103.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.103.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.104.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.104.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.105.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.105.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.106.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.106.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.107.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.107.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.108.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.108.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.109.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.109.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.110.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.110.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.111.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.111.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.112.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.112.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.113.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.113.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.114.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.114.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.115.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.115.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.116.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.116.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.117.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.117.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.118.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.118.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.119.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.119.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.120.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.120.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.121.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.121.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.122.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.122.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.123.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.123.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.124.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.124.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.125.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.125.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.126.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.126.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.127.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.127.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.0.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.1.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.2.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.3.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.4.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.5.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.6.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.7.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.8.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.9.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.10.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.11.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.12.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.13.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.14.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.15.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.16.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.17.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.18.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.19.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.20.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.21.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.22.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.23.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.24.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.25.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.26.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.27.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.28.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.29.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.30.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.31.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.32.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.33.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.34.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.35.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.36.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.37.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.38.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.39.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.40.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.41.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.42.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.43.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.44.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.45.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.46.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.47.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.48.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.49.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.50.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.51.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.52.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.53.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.54.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.55.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.56.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.57.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.58.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.59.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.60.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.61.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.62.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.63.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.64.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.65.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.66.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.67.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.68.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.69.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.70.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.71.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.72.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.73.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.74.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.75.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.76.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.77.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.78.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.79.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.80.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.81.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.82.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.83.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.84.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.85.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.86.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.87.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.88.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.89.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.90.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.91.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.92.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.93.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.94.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.95.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.96.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.97.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.98.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.99.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.100.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.101.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.102.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.103.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.104.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.105.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.106.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.107.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.108.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.109.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.110.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.111.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.112.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.113.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.114.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.115.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.116.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.117.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.118.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.119.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.120.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.121.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.122.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.123.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.124.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.125.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.126.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.experts.127.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.q_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.k_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.v_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.gate.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.o_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.shared_experts.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.shared_experts.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.shared_experts.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.mlp.gate.e_score_correction_bias": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.q_proj.bias": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.k_proj.bias": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.self_attn.v_proj.bias": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.post_attention_layernorm.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.7.input_layernorm.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.0.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.0.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.1.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.1.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.2.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.2.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.3.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.3.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.4.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.4.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.5.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.5.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.6.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.6.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.7.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.7.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.8.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.8.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.9.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.9.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.10.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.10.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.11.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.11.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.12.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.12.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.13.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.13.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.14.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.14.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.15.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.15.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.16.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.16.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.17.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.17.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.18.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.18.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.19.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.19.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.20.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.20.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.21.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.21.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.22.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.22.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.23.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.23.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.24.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.24.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.25.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.25.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.26.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.26.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.27.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.27.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.28.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.28.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.29.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.29.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.30.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.30.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.31.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.31.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.32.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.32.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.33.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.33.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.34.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.34.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.35.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.35.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.36.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.36.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.37.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.37.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.38.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.38.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.39.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.39.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.40.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.40.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.41.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.41.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.42.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.42.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.43.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.43.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.44.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.44.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.45.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.45.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.46.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.46.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.47.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.47.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.48.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.48.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.49.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.49.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.50.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.50.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.51.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.51.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.52.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.52.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.53.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.53.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.54.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.54.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.55.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.55.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.56.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.56.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.57.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.57.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.58.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.58.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.59.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.59.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.60.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.60.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.61.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.61.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.62.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.62.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.63.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.63.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.64.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.64.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.65.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.65.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.66.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.66.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.67.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.67.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.68.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.68.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.69.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.69.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.70.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.70.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.71.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.71.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.72.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.72.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.73.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.73.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.74.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.74.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.75.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.75.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.76.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.76.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.77.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.77.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.78.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.78.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.79.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.79.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.80.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.80.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.81.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.81.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.82.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.82.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.83.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.83.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.84.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.84.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.85.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.85.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.86.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.86.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.87.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.87.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.88.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.88.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.89.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.89.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.90.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.90.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.91.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.91.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.92.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.92.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.93.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.93.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.94.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.94.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.95.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.95.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.96.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.96.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.97.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.97.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.98.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.98.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.99.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.99.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.100.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.100.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.101.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.101.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.102.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.102.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.103.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.103.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.104.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.104.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.105.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.105.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.106.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.106.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.107.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.107.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.108.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.108.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.109.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.109.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.110.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.110.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.111.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.111.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.112.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.112.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.113.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.113.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.114.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.114.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.115.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.115.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.116.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.116.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.117.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.117.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.118.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.118.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.119.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.119.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.120.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.120.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.121.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.121.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.122.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.122.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.123.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.123.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.124.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.124.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.125.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.125.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.126.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.126.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.127.gate_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.127.up_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.0.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.1.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.2.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.3.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.4.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.5.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.6.down_proj.weight": "model-00007-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.7.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.8.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.9.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.10.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.11.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.12.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.13.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.14.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.15.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.16.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.17.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.18.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.19.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.20.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.21.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.22.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.23.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.24.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.25.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.26.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.27.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.28.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.29.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.30.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.31.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.32.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.33.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.34.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.35.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.36.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.37.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.38.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.39.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.40.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.41.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.42.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.43.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.44.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.45.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.46.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.47.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.48.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.49.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.50.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.51.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.52.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.53.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.54.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.55.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.56.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.57.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.58.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.59.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.60.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.61.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.62.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.63.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.64.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.65.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.66.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.67.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.68.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.69.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.70.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.71.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.72.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.73.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.74.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.75.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.76.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.77.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.78.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.79.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.80.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.81.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.82.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.83.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.84.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.85.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.86.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.87.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.88.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.89.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.90.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.91.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.92.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.93.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.94.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.95.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.96.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.97.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.98.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.99.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.100.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.101.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.102.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.103.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.104.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.105.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.106.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.107.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.108.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.109.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.110.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.111.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.112.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.113.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.114.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.115.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.116.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.117.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.118.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.119.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.120.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.121.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.122.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.123.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.124.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.125.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.126.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.experts.127.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.post_attention_layernorm.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.q_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.k_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.v_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.gate.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.o_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.shared_experts.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.shared_experts.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.shared_experts.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.input_layernorm.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.mlp.gate.e_score_correction_bias": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.q_proj.bias": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.k_proj.bias": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.8.self_attn.v_proj.bias": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.0.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.0.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.1.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.1.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.2.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.2.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.3.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.3.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.4.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.4.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.5.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.5.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.6.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.6.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.7.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.7.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.8.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.8.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.9.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.9.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.10.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.10.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.11.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.11.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.12.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.12.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.13.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.13.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.14.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.14.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.15.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.15.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.16.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.16.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.17.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.17.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.18.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.18.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.19.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.19.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.20.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.20.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.21.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.21.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.22.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.22.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.23.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.23.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.24.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.24.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.25.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.25.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.26.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.26.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.27.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.27.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.28.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.28.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.29.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.29.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.30.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.30.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.31.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.31.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.32.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.32.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.33.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.33.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.34.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.34.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.35.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.35.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.36.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.36.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.37.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.37.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.38.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.38.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.39.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.39.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.40.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.40.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.41.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.41.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.42.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.42.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.43.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.43.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.44.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.44.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.45.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.45.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.46.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.46.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.47.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.47.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.48.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.48.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.49.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.49.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.50.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.50.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.51.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.51.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.52.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.52.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.53.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.53.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.54.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.54.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.55.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.55.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.56.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.56.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.57.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.57.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.58.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.58.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.59.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.59.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.60.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.60.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.61.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.61.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.62.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.62.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.63.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.63.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.64.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.64.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.65.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.65.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.66.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.66.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.67.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.67.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.68.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.68.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.69.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.69.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.70.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.70.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.71.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.71.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.72.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.72.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.73.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.73.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.74.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.74.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.75.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.75.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.76.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.76.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.77.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.77.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.78.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.78.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.79.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.79.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.80.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.80.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.81.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.81.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.82.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.82.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.83.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.83.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.84.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.84.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.85.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.85.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.86.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.86.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.87.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.87.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.88.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.88.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.89.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.89.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.90.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.90.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.91.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.91.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.92.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.92.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.93.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.93.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.94.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.94.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.95.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.95.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.96.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.96.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.97.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.97.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.98.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.98.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.99.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.99.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.100.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.100.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.101.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.101.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.102.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.102.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.103.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.103.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.104.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.104.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.105.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.105.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.106.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.106.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.107.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.107.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.108.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.108.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.109.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.109.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.110.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.110.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.111.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.111.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.112.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.112.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.113.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.113.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.114.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.114.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.115.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.115.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.116.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.116.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.117.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.117.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.118.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.118.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.119.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.119.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.120.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.120.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.121.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.121.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.122.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.122.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.123.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.123.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.124.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.124.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.125.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.125.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.126.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.126.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.127.gate_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.127.up_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.0.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.1.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.2.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.3.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.4.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.5.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.6.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.7.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.8.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.9.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.10.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.11.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.12.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.13.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.14.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.15.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.16.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.17.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.18.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.19.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.20.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.21.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.22.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.23.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.24.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.25.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.26.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.27.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.28.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.29.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.30.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.31.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.32.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.33.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.34.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.35.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.36.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.37.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.38.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.39.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.40.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.41.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.42.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.43.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.44.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.45.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.46.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.47.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.48.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.49.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.50.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.51.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.52.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.53.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.54.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.55.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.56.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.57.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.58.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.59.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.60.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.61.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.62.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.63.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.64.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.65.down_proj.weight": "model-00008-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.66.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.67.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.68.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.69.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.70.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.71.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.72.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.73.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.74.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.75.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.76.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.77.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.78.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.79.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.80.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.81.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.82.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.83.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.84.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.85.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.86.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.87.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.88.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.89.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.90.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.91.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.92.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.93.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.94.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.95.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.96.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.97.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.98.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.99.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.100.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.101.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.102.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.103.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.104.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.105.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.106.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.107.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.108.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.109.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.110.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.111.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.112.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.113.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.114.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.115.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.116.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.117.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.118.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.119.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.120.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.121.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.122.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.123.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.124.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.125.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.126.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.experts.127.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.q_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.k_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.v_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.gate.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.o_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.shared_experts.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.shared_experts.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.q_proj.bias": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.k_proj.bias": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.self_attn.v_proj.bias": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.post_attention_layernorm.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.shared_experts.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.input_layernorm.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.9.mlp.gate.e_score_correction_bias": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.0.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.0.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.1.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.1.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.2.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.2.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.3.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.3.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.4.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.4.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.5.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.5.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.6.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.6.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.7.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.7.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.8.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.8.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.9.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.9.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.10.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.10.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.11.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.11.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.12.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.12.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.13.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.13.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.14.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.14.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.15.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.15.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.16.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.16.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.17.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.17.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.18.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.18.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.19.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.19.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.20.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.20.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.21.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.21.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.22.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.22.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.23.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.23.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.24.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.24.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.25.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.25.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.26.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.26.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.27.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.27.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.28.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.28.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.29.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.29.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.30.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.30.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.31.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.31.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.32.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.32.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.33.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.33.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.34.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.34.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.35.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.35.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.36.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.36.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.37.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.37.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.38.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.38.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.39.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.39.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.40.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.40.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.41.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.41.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.42.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.42.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.43.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.43.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.44.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.44.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.45.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.45.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.46.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.46.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.47.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.47.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.48.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.48.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.49.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.49.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.50.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.50.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.51.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.51.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.52.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.52.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.53.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.53.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.54.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.54.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.55.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.55.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.56.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.56.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.57.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.57.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.58.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.58.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.59.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.59.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.60.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.60.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.61.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.61.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.62.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.62.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.63.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.63.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.64.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.64.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.65.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.65.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.66.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.66.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.67.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.67.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.68.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.68.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.69.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.69.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.70.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.70.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.71.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.71.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.72.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.72.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.73.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.73.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.74.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.74.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.75.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.75.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.76.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.76.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.77.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.77.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.78.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.78.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.79.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.79.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.80.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.80.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.81.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.81.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.82.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.82.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.83.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.83.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.84.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.84.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.85.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.85.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.86.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.86.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.87.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.87.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.88.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.88.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.89.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.89.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.90.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.90.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.91.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.91.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.92.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.92.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.93.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.93.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.94.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.94.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.95.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.95.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.96.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.96.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.97.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.97.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.98.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.98.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.99.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.99.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.100.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.100.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.101.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.101.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.102.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.102.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.103.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.103.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.104.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.104.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.105.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.105.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.106.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.106.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.107.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.107.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.108.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.108.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.109.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.109.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.110.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.110.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.111.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.111.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.112.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.112.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.113.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.113.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.114.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.114.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.115.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.115.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.116.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.116.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.117.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.117.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.118.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.118.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.119.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.119.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.120.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.120.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.121.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.121.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.122.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.122.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.123.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.123.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.124.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.124.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.125.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.125.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.126.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.126.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.127.gate_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.127.up_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.0.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.1.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.2.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.3.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.4.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.5.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.6.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.7.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.8.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.9.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.10.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.11.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.12.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.13.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.14.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.15.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.16.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.17.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.18.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.19.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.20.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.21.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.22.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.23.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.24.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.25.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.26.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.27.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.28.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.29.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.30.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.31.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.32.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.33.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.34.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.35.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.36.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.37.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.38.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.39.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.40.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.41.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.42.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.43.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.44.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.45.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.46.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.47.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.48.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.49.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.50.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.51.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.52.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.53.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.54.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.55.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.56.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.57.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.58.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.59.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.60.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.61.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.62.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.63.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.64.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.65.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.66.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.67.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.68.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.69.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.70.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.71.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.72.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.73.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.74.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.75.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.76.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.77.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.78.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.79.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.80.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.81.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.82.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.83.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.84.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.85.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.86.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.87.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.88.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.89.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.90.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.91.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.92.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.93.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.94.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.95.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.96.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.97.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.98.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.99.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.100.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.101.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.102.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.103.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.104.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.105.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.106.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.107.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.108.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.109.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.110.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.111.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.112.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.113.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.114.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.115.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.116.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.117.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.118.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.119.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.120.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.121.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.122.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.123.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.124.down_proj.weight": "model-00009-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.125.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.126.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.experts.127.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.q_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.k_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.v_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.shared_experts.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.shared_experts.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.shared_experts.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.gate.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.o_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.input_layernorm.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.q_proj.bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.k_proj.bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.self_attn.v_proj.bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.post_attention_layernorm.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.10.mlp.gate.e_score_correction_bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.0.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.0.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.1.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.1.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.2.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.2.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.3.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.3.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.4.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.4.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.5.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.5.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.6.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.6.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.7.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.7.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.8.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.8.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.9.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.9.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.10.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.10.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.11.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.11.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.12.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.12.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.13.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.13.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.14.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.14.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.15.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.15.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.16.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.16.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.17.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.17.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.18.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.18.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.19.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.19.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.20.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.20.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.21.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.21.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.22.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.22.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.23.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.23.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.24.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.24.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.25.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.25.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.26.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.26.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.27.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.27.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.28.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.28.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.29.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.29.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.30.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.30.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.31.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.31.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.32.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.32.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.33.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.33.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.34.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.34.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.35.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.35.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.36.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.36.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.37.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.37.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.38.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.38.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.39.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.39.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.40.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.40.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.41.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.41.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.42.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.42.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.43.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.43.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.44.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.44.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.45.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.45.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.46.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.46.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.47.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.47.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.48.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.48.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.49.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.49.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.50.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.50.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.51.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.51.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.52.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.52.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.53.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.53.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.54.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.54.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.55.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.55.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.56.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.56.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.57.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.57.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.58.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.58.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.59.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.59.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.60.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.60.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.61.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.61.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.62.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.62.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.63.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.63.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.64.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.64.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.65.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.65.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.66.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.66.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.67.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.67.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.68.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.68.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.69.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.69.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.70.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.70.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.71.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.71.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.72.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.72.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.73.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.73.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.74.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.74.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.75.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.75.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.76.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.76.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.77.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.77.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.78.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.78.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.79.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.79.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.80.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.80.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.81.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.81.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.82.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.82.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.83.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.83.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.84.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.84.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.85.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.85.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.86.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.86.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.87.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.87.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.88.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.88.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.89.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.89.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.90.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.90.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.91.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.91.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.92.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.92.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.93.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.93.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.94.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.94.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.95.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.95.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.96.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.96.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.97.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.97.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.98.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.98.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.99.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.99.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.100.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.100.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.101.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.101.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.102.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.102.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.103.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.103.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.104.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.104.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.105.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.105.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.106.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.106.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.107.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.107.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.108.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.108.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.109.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.109.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.110.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.110.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.111.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.111.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.112.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.112.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.113.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.113.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.114.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.114.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.115.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.115.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.116.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.116.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.117.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.117.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.118.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.118.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.119.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.119.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.120.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.120.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.121.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.121.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.122.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.122.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.123.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.123.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.124.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.124.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.125.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.125.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.126.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.126.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.127.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.127.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.0.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.1.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.2.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.3.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.4.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.5.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.6.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.7.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.8.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.9.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.10.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.11.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.12.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.13.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.14.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.15.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.16.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.17.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.18.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.19.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.20.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.21.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.22.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.23.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.24.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.25.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.26.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.27.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.28.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.29.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.30.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.31.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.32.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.33.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.34.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.35.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.36.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.37.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.38.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.39.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.40.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.41.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.42.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.43.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.44.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.45.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.46.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.47.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.48.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.49.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.50.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.51.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.52.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.53.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.54.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.55.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.56.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.57.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.58.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.59.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.60.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.61.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.62.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.63.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.64.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.65.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.66.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.67.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.68.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.69.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.70.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.71.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.72.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.73.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.74.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.75.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.76.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.77.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.78.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.79.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.80.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.81.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.82.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.83.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.84.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.85.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.86.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.87.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.88.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.89.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.90.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.91.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.92.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.93.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.94.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.95.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.96.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.97.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.98.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.99.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.100.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.101.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.102.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.103.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.104.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.105.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.106.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.107.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.108.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.109.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.110.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.111.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.112.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.113.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.114.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.115.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.116.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.117.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.118.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.119.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.120.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.121.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.122.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.123.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.124.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.125.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.126.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.experts.127.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.q_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.k_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.v_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.gate.e_score_correction_bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.shared_experts.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.shared_experts.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.gate.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.mlp.shared_experts.down_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.o_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.input_layernorm.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.q_proj.bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.k_proj.bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.self_attn.v_proj.bias": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.11.post_attention_layernorm.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.0.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.0.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.1.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.1.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.2.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.2.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.3.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.3.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.4.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.4.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.5.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.5.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.6.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.6.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.7.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.7.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.8.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.8.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.9.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.9.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.10.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.10.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.11.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.11.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.12.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.12.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.13.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.13.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.14.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.14.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.15.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.15.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.16.gate_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.16.up_proj.weight": "model-00010-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.17.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.17.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.18.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.18.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.19.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.19.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.20.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.20.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.21.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.21.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.22.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.22.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.23.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.23.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.24.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.24.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.25.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.25.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.26.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.26.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.27.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.27.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.28.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.28.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.29.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.29.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.30.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.30.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.31.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.31.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.32.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.32.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.33.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.33.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.34.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.34.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.35.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.35.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.36.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.36.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.37.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.37.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.38.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.38.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.39.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.39.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.40.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.40.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.41.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.41.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.42.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.42.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.43.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.43.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.44.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.44.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.45.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.45.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.46.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.46.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.47.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.47.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.48.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.48.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.49.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.49.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.50.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.50.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.51.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.51.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.52.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.52.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.53.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.53.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.54.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.54.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.55.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.55.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.56.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.56.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.57.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.57.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.58.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.58.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.59.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.59.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.60.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.60.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.61.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.61.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.62.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.62.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.63.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.63.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.64.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.64.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.65.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.65.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.66.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.66.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.67.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.67.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.68.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.68.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.69.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.69.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.70.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.70.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.71.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.71.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.72.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.72.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.73.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.73.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.74.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.74.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.75.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.75.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.76.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.76.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.77.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.77.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.78.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.78.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.79.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.79.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.80.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.80.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.81.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.81.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.82.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.82.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.83.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.83.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.84.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.84.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.85.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.85.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.86.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.86.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.87.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.87.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.88.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.88.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.89.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.89.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.90.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.90.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.91.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.91.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.92.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.92.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.93.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.93.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.94.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.94.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.95.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.95.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.96.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.96.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.97.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.97.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.98.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.98.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.99.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.99.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.100.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.100.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.101.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.101.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.102.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.102.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.103.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.103.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.104.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.104.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.105.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.105.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.106.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.106.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.107.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.107.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.108.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.108.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.109.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.109.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.110.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.110.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.111.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.111.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.112.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.112.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.113.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.113.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.114.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.114.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.115.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.115.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.116.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.116.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.117.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.117.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.118.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.118.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.119.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.119.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.120.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.120.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.121.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.121.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.122.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.122.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.123.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.123.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.124.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.124.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.125.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.125.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.126.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.126.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.127.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.127.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.0.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.1.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.2.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.3.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.4.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.5.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.6.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.7.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.8.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.9.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.10.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.11.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.12.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.13.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.14.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.15.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.16.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.17.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.18.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.19.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.20.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.21.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.22.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.23.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.24.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.25.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.26.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.27.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.28.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.29.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.30.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.31.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.32.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.33.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.34.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.35.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.36.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.37.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.38.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.39.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.40.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.41.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.42.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.43.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.44.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.45.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.46.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.47.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.48.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.49.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.50.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.51.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.52.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.53.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.54.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.55.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.56.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.57.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.58.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.59.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.60.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.61.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.62.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.63.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.64.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.65.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.66.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.67.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.68.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.69.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.70.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.71.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.72.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.73.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.74.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.75.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.76.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.77.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.78.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.79.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.80.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.81.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.82.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.83.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.84.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.85.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.86.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.87.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.88.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.89.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.90.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.91.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.92.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.93.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.94.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.95.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.96.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.97.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.98.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.99.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.100.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.101.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.102.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.103.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.104.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.105.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.106.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.107.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.108.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.109.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.110.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.111.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.112.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.113.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.114.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.115.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.116.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.117.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.118.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.119.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.120.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.121.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.122.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.123.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.124.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.125.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.126.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.experts.127.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.q_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.k_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.v_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.gate.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.shared_experts.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.shared_experts.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.shared_experts.down_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.o_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.mlp.gate.e_score_correction_bias": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.q_proj.bias": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.k_proj.bias": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.self_attn.v_proj.bias": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.post_attention_layernorm.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.12.input_layernorm.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.0.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.0.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.1.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.1.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.2.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.2.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.3.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.3.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.4.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.4.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.5.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.5.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.6.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.6.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.7.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.7.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.8.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.8.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.9.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.9.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.10.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.10.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.11.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.11.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.12.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.12.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.13.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.13.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.14.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.14.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.15.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.15.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.16.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.16.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.17.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.17.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.18.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.18.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.19.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.19.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.20.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.20.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.21.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.21.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.22.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.22.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.23.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.23.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.24.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.24.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.25.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.25.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.26.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.26.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.27.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.27.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.28.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.28.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.29.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.29.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.30.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.30.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.31.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.31.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.32.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.32.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.33.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.33.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.34.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.34.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.35.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.35.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.36.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.36.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.37.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.37.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.38.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.38.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.39.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.39.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.40.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.40.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.41.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.41.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.42.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.42.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.43.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.43.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.44.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.44.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.45.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.45.up_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.46.gate_proj.weight": "model-00011-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.46.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.47.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.47.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.48.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.48.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.49.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.49.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.50.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.50.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.51.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.51.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.52.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.52.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.53.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.53.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.54.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.54.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.55.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.55.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.56.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.56.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.57.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.57.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.58.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.58.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.59.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.59.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.60.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.60.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.61.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.61.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.62.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.62.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.63.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.63.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.64.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.64.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.65.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.65.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.66.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.66.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.67.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.67.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.68.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.68.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.69.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.69.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.70.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.70.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.71.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.71.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.72.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.72.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.73.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.73.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.74.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.74.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.75.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.75.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.76.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.76.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.77.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.77.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.78.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.78.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.79.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.79.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.80.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.80.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.81.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.81.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.82.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.82.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.83.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.83.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.84.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.84.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.85.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.85.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.86.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.86.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.87.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.87.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.88.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.88.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.89.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.89.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.90.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.90.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.91.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.91.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.92.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.92.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.93.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.93.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.94.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.94.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.95.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.95.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.96.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.96.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.97.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.97.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.98.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.98.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.99.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.99.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.100.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.100.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.101.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.101.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.102.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.102.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.103.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.103.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.104.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.104.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.105.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.105.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.106.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.106.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.107.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.107.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.108.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.108.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.109.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.109.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.110.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.110.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.111.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.111.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.112.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.112.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.113.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.113.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.114.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.114.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.115.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.115.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.116.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.116.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.117.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.117.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.118.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.118.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.119.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.119.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.120.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.120.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.121.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.121.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.122.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.122.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.123.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.123.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.124.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.124.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.125.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.125.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.126.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.126.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.127.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.127.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.0.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.1.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.2.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.3.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.4.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.5.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.6.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.7.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.8.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.9.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.10.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.11.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.12.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.13.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.14.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.15.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.16.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.17.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.18.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.19.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.20.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.21.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.22.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.23.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.24.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.25.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.26.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.27.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.28.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.29.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.30.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.31.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.32.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.33.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.34.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.35.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.36.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.37.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.38.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.39.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.40.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.41.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.42.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.43.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.44.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.45.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.46.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.47.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.48.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.49.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.50.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.51.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.52.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.53.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.54.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.55.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.56.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.57.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.58.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.59.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.60.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.61.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.62.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.63.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.64.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.65.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.66.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.67.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.68.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.69.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.70.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.71.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.72.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.73.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.74.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.75.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.76.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.77.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.78.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.79.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.80.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.81.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.82.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.83.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.84.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.85.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.86.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.87.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.88.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.89.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.90.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.91.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.92.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.93.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.94.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.95.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.96.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.97.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.98.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.99.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.100.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.101.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.102.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.103.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.104.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.105.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.106.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.107.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.108.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.109.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.110.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.111.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.112.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.113.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.114.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.115.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.116.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.117.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.118.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.119.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.120.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.121.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.122.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.123.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.124.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.125.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.126.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.experts.127.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.q_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.k_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.v_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.post_attention_layernorm.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.gate.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.shared_experts.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.shared_experts.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.o_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.input_layernorm.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.shared_experts.down_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.q_proj.bias": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.k_proj.bias": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.self_attn.v_proj.bias": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.13.mlp.gate.e_score_correction_bias": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.0.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.0.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.1.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.1.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.2.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.2.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.3.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.3.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.4.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.4.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.5.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.5.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.6.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.6.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.7.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.7.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.8.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.8.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.9.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.9.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.10.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.10.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.11.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.11.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.12.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.12.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.13.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.13.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.14.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.14.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.15.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.15.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.16.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.16.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.17.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.17.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.18.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.18.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.19.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.19.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.20.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.20.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.21.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.21.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.22.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.22.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.23.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.23.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.24.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.24.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.25.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.25.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.26.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.26.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.27.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.27.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.28.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.28.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.29.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.29.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.30.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.30.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.31.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.31.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.32.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.32.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.33.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.33.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.34.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.34.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.35.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.35.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.36.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.36.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.37.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.37.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.38.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.38.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.39.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.39.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.40.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.40.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.41.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.41.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.42.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.42.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.43.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.43.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.44.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.44.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.45.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.45.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.46.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.46.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.47.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.47.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.48.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.48.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.49.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.49.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.50.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.50.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.51.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.51.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.52.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.52.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.53.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.53.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.54.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.54.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.55.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.55.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.56.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.56.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.57.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.57.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.58.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.58.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.59.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.59.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.60.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.60.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.61.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.61.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.62.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.62.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.63.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.63.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.64.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.64.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.65.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.65.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.66.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.66.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.67.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.67.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.68.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.68.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.69.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.69.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.70.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.70.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.71.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.71.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.72.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.72.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.73.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.73.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.74.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.74.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.75.gate_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.75.up_proj.weight": "model-00012-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.76.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.76.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.77.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.77.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.78.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.78.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.79.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.79.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.80.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.80.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.81.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.81.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.82.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.82.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.83.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.83.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.84.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.84.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.85.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.85.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.86.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.86.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.87.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.87.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.88.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.88.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.89.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.89.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.90.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.90.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.91.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.91.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.92.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.92.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.93.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.93.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.94.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.94.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.95.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.95.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.96.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.96.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.97.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.97.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.98.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.98.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.99.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.99.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.100.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.100.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.101.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.101.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.102.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.102.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.103.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.103.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.104.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.104.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.105.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.105.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.106.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.106.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.107.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.107.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.108.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.108.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.109.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.109.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.110.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.110.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.111.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.111.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.112.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.112.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.113.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.113.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.114.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.114.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.115.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.115.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.116.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.116.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.117.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.117.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.118.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.118.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.119.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.119.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.120.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.120.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.121.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.121.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.122.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.122.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.123.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.123.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.124.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.124.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.125.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.125.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.126.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.126.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.127.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.127.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.0.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.1.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.2.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.3.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.4.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.5.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.6.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.7.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.8.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.9.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.10.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.11.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.12.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.13.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.14.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.15.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.16.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.17.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.18.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.19.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.20.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.21.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.22.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.23.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.24.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.25.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.26.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.27.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.28.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.29.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.30.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.31.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.32.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.33.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.34.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.35.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.36.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.37.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.38.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.39.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.40.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.41.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.42.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.43.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.44.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.45.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.46.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.47.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.48.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.49.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.50.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.51.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.52.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.53.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.54.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.55.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.56.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.57.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.58.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.59.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.60.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.61.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.62.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.63.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.64.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.65.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.66.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.67.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.68.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.69.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.70.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.71.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.72.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.73.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.74.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.75.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.76.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.77.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.78.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.79.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.80.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.81.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.82.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.83.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.84.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.85.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.86.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.87.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.88.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.89.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.90.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.91.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.92.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.93.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.94.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.95.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.96.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.97.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.98.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.99.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.100.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.101.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.102.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.103.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.104.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.105.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.106.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.107.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.108.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.109.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.110.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.111.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.112.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.113.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.114.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.115.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.116.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.117.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.118.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.119.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.120.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.121.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.122.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.123.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.124.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.125.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.126.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.experts.127.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.q_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.k_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.v_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.gate.e_score_correction_bias": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.gate.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.o_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.q_proj.bias": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.k_proj.bias": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.self_attn.v_proj.bias": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.shared_experts.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.shared_experts.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.post_attention_layernorm.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.mlp.shared_experts.down_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.14.input_layernorm.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.0.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.0.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.1.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.1.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.2.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.2.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.3.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.3.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.4.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.4.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.5.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.5.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.6.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.6.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.7.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.7.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.8.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.8.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.9.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.9.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.10.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.10.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.11.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.11.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.12.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.12.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.13.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.13.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.14.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.14.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.15.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.15.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.16.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.16.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.17.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.17.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.18.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.18.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.19.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.19.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.20.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.20.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.21.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.21.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.22.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.22.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.23.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.23.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.24.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.24.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.25.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.25.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.26.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.26.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.27.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.27.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.28.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.28.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.29.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.29.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.30.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.30.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.31.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.31.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.32.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.32.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.33.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.33.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.34.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.34.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.35.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.35.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.36.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.36.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.37.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.37.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.38.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.38.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.39.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.39.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.40.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.40.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.41.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.41.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.42.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.42.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.43.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.43.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.44.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.44.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.45.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.45.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.46.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.46.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.47.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.47.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.48.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.48.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.49.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.49.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.50.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.50.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.51.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.51.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.52.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.52.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.53.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.53.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.54.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.54.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.55.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.55.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.56.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.56.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.57.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.57.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.58.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.58.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.59.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.59.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.60.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.60.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.61.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.61.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.62.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.62.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.63.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.63.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.64.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.64.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.65.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.65.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.66.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.66.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.67.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.67.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.68.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.68.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.69.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.69.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.70.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.70.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.71.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.71.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.72.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.72.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.73.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.73.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.74.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.74.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.75.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.75.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.76.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.76.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.77.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.77.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.78.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.78.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.79.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.79.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.80.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.80.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.81.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.81.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.82.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.82.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.83.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.83.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.84.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.84.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.85.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.85.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.86.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.86.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.87.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.87.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.88.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.88.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.89.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.89.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.90.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.90.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.91.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.91.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.92.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.92.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.93.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.93.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.94.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.94.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.95.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.95.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.96.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.96.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.97.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.97.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.98.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.98.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.99.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.99.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.100.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.100.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.101.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.101.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.102.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.102.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.103.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.103.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.104.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.104.up_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.105.gate_proj.weight": "model-00013-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.105.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.106.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.106.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.107.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.107.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.108.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.108.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.109.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.109.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.110.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.110.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.111.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.111.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.112.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.112.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.113.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.113.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.114.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.114.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.115.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.115.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.116.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.116.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.117.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.117.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.118.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.118.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.119.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.119.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.120.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.120.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.121.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.121.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.122.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.122.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.123.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.123.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.124.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.124.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.125.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.125.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.126.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.126.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.127.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.127.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.0.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.1.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.2.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.3.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.4.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.5.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.6.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.7.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.8.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.9.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.10.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.11.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.12.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.13.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.14.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.15.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.16.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.17.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.18.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.19.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.20.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.21.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.22.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.23.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.24.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.25.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.26.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.27.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.28.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.29.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.30.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.31.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.32.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.33.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.34.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.35.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.36.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.37.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.38.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.39.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.40.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.41.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.42.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.43.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.44.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.45.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.46.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.47.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.48.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.49.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.50.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.51.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.52.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.53.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.54.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.55.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.56.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.57.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.58.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.59.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.60.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.61.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.62.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.63.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.64.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.65.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.66.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.67.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.68.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.69.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.70.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.71.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.72.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.73.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.74.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.75.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.76.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.77.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.78.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.79.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.80.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.81.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.82.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.83.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.84.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.85.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.86.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.87.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.88.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.89.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.90.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.91.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.92.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.93.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.94.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.95.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.96.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.97.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.98.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.99.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.100.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.101.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.102.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.103.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.104.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.105.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.106.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.107.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.108.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.109.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.110.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.111.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.112.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.113.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.114.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.115.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.116.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.117.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.118.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.119.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.120.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.121.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.122.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.123.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.124.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.125.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.126.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.experts.127.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.shared_experts.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.shared_experts.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.shared_experts.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.post_attention_layernorm.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.q_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.k_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.v_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.gate.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.o_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.input_layernorm.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.mlp.gate.e_score_correction_bias": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.q_proj.bias": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.k_proj.bias": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.15.self_attn.v_proj.bias": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.0.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.0.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.1.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.1.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.2.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.2.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.3.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.3.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.4.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.4.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.5.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.5.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.6.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.6.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.7.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.7.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.8.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.8.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.9.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.9.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.10.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.10.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.11.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.11.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.12.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.12.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.13.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.13.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.14.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.14.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.15.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.15.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.16.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.16.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.17.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.17.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.18.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.18.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.19.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.19.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.20.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.20.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.21.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.21.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.22.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.22.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.23.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.23.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.24.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.24.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.25.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.25.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.26.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.26.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.27.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.27.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.28.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.28.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.29.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.29.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.30.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.30.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.31.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.31.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.32.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.32.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.33.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.33.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.34.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.34.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.35.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.35.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.36.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.36.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.37.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.37.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.38.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.38.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.39.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.39.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.40.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.40.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.41.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.41.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.42.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.42.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.43.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.43.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.44.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.44.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.45.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.45.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.46.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.46.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.47.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.47.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.48.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.48.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.49.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.49.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.50.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.50.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.51.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.51.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.52.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.52.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.53.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.53.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.54.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.54.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.55.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.55.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.56.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.56.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.57.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.57.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.58.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.58.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.59.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.59.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.60.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.60.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.61.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.61.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.62.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.62.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.63.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.63.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.64.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.64.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.65.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.65.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.66.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.66.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.67.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.67.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.68.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.68.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.69.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.69.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.70.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.70.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.71.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.71.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.72.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.72.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.73.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.73.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.74.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.74.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.75.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.75.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.76.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.76.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.77.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.77.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.78.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.78.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.79.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.79.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.80.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.80.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.81.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.81.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.82.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.82.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.83.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.83.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.84.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.84.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.85.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.85.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.86.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.86.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.87.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.87.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.88.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.88.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.89.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.89.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.90.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.90.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.91.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.91.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.92.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.92.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.93.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.93.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.94.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.94.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.95.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.95.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.96.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.96.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.97.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.97.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.98.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.98.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.99.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.99.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.100.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.100.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.101.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.101.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.102.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.102.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.103.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.103.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.104.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.104.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.105.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.105.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.106.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.106.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.107.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.107.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.108.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.108.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.109.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.109.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.110.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.110.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.111.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.111.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.112.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.112.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.113.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.113.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.114.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.114.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.115.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.115.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.116.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.116.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.117.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.117.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.118.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.118.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.119.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.119.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.120.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.120.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.121.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.121.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.122.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.122.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.123.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.123.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.124.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.124.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.125.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.125.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.126.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.126.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.127.gate_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.127.up_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.0.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.1.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.2.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.3.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.4.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.5.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.6.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.7.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.8.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.9.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.10.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.11.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.12.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.13.down_proj.weight": "model-00014-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.14.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.15.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.16.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.17.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.18.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.19.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.20.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.21.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.22.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.23.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.24.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.25.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.26.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.27.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.28.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.29.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.30.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.31.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.32.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.33.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.34.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.35.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.36.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.37.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.38.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.39.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.40.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.41.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.42.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.43.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.44.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.45.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.46.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.47.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.48.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.49.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.50.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.51.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.52.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.53.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.54.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.55.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.56.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.57.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.58.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.59.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.60.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.61.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.62.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.63.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.64.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.65.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.66.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.67.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.68.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.69.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.70.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.71.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.72.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.73.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.74.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.75.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.76.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.77.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.78.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.79.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.80.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.81.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.82.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.83.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.84.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.85.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.86.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.87.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.88.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.89.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.90.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.91.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.92.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.93.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.94.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.95.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.96.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.97.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.98.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.99.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.100.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.101.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.102.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.103.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.104.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.105.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.106.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.107.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.108.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.109.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.110.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.111.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.112.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.113.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.114.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.115.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.116.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.117.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.118.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.119.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.120.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.121.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.122.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.123.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.124.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.125.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.126.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.experts.127.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.shared_experts.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.shared_experts.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.shared_experts.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.q_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.k_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.v_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.gate.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.o_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.post_attention_layernorm.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.input_layernorm.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.q_proj.bias": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.k_proj.bias": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.self_attn.v_proj.bias": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.16.mlp.gate.e_score_correction_bias": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.0.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.0.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.1.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.1.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.2.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.2.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.3.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.3.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.4.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.4.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.5.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.5.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.6.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.6.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.7.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.7.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.8.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.8.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.9.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.9.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.10.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.10.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.11.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.11.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.12.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.12.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.13.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.13.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.14.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.14.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.15.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.15.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.16.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.16.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.17.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.17.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.18.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.18.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.19.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.19.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.20.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.20.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.21.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.21.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.22.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.22.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.23.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.23.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.24.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.24.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.25.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.25.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.26.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.26.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.27.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.27.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.28.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.28.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.29.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.29.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.30.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.30.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.31.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.31.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.32.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.32.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.33.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.33.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.34.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.34.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.35.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.35.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.36.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.36.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.37.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.37.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.38.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.38.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.39.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.39.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.40.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.40.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.41.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.41.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.42.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.42.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.43.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.43.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.44.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.44.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.45.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.45.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.46.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.46.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.47.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.47.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.48.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.48.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.49.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.49.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.50.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.50.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.51.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.51.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.52.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.52.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.53.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.53.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.54.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.54.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.55.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.55.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.56.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.56.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.57.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.57.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.58.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.58.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.59.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.59.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.60.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.60.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.61.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.61.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.62.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.62.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.63.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.63.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.64.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.64.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.65.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.65.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.66.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.66.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.67.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.67.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.68.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.68.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.69.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.69.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.70.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.70.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.71.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.71.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.72.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.72.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.73.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.73.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.74.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.74.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.75.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.75.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.76.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.76.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.77.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.77.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.78.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.78.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.79.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.79.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.80.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.80.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.81.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.81.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.82.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.82.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.83.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.83.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.84.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.84.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.85.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.85.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.86.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.86.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.87.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.87.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.88.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.88.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.89.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.89.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.90.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.90.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.91.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.91.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.92.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.92.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.93.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.93.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.94.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.94.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.95.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.95.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.96.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.96.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.97.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.97.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.98.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.98.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.99.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.99.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.100.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.100.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.101.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.101.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.102.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.102.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.103.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.103.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.104.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.104.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.105.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.105.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.106.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.106.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.107.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.107.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.108.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.108.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.109.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.109.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.110.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.110.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.111.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.111.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.112.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.112.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.113.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.113.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.114.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.114.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.115.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.115.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.116.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.116.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.117.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.117.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.118.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.118.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.119.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.119.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.120.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.120.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.121.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.121.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.122.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.122.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.123.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.123.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.124.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.124.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.125.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.125.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.126.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.126.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.127.gate_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.127.up_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.0.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.1.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.2.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.3.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.4.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.5.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.6.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.7.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.8.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.9.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.10.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.11.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.12.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.13.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.14.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.15.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.16.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.17.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.18.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.19.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.20.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.21.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.22.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.23.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.24.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.25.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.26.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.27.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.28.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.29.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.30.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.31.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.32.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.33.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.34.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.35.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.36.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.37.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.38.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.39.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.40.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.41.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.42.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.43.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.44.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.45.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.46.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.47.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.48.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.49.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.50.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.51.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.52.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.53.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.54.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.55.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.56.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.57.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.58.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.59.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.60.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.61.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.62.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.63.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.64.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.65.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.66.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.67.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.68.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.69.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.70.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.71.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.72.down_proj.weight": "model-00015-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.73.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.74.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.75.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.76.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.77.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.78.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.79.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.80.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.81.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.82.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.83.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.84.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.85.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.86.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.87.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.88.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.89.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.90.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.91.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.92.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.93.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.94.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.95.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.96.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.97.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.98.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.99.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.100.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.101.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.102.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.103.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.104.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.105.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.106.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.107.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.108.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.109.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.110.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.111.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.112.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.113.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.114.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.115.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.116.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.117.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.118.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.119.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.120.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.121.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.122.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.123.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.124.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.125.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.126.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.experts.127.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.gate.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.post_attention_layernorm.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.gate.e_score_correction_bias": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.shared_experts.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.shared_experts.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.q_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.k_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.v_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.mlp.shared_experts.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.input_layernorm.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.q_proj.bias": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.k_proj.bias": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.v_proj.bias": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.17.self_attn.o_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.0.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.0.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.1.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.1.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.2.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.2.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.3.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.3.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.4.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.4.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.5.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.5.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.6.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.6.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.7.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.7.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.8.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.8.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.9.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.9.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.10.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.10.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.11.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.11.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.12.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.12.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.13.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.13.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.14.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.14.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.15.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.15.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.16.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.16.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.17.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.17.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.18.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.18.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.19.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.19.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.20.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.20.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.21.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.21.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.22.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.22.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.23.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.23.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.24.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.24.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.25.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.25.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.26.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.26.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.27.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.27.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.28.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.28.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.29.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.29.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.30.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.30.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.31.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.31.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.32.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.32.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.33.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.33.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.34.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.34.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.35.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.35.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.36.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.36.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.37.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.37.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.38.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.38.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.39.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.39.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.40.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.40.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.41.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.41.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.42.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.42.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.43.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.43.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.44.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.44.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.45.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.45.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.46.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.46.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.47.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.47.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.48.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.48.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.49.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.49.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.50.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.50.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.51.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.51.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.52.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.52.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.53.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.53.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.54.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.54.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.55.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.55.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.56.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.56.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.57.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.57.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.58.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.58.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.59.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.59.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.60.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.60.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.61.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.61.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.62.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.62.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.63.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.63.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.64.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.64.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.65.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.65.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.66.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.66.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.67.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.67.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.68.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.68.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.69.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.69.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.70.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.70.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.71.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.71.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.72.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.72.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.73.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.73.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.74.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.74.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.75.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.75.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.76.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.76.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.77.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.77.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.78.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.78.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.79.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.79.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.80.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.80.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.81.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.81.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.82.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.82.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.83.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.83.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.84.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.84.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.85.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.85.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.86.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.86.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.87.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.87.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.88.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.88.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.89.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.89.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.90.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.90.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.91.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.91.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.92.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.92.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.93.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.93.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.94.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.94.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.95.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.95.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.96.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.96.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.97.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.97.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.98.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.98.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.99.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.99.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.100.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.100.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.101.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.101.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.102.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.102.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.103.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.103.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.104.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.104.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.105.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.105.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.106.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.106.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.107.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.107.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.108.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.108.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.109.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.109.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.110.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.110.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.111.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.111.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.112.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.112.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.113.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.113.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.114.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.114.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.115.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.115.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.116.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.116.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.117.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.117.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.118.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.118.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.119.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.119.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.120.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.120.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.121.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.121.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.122.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.122.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.123.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.123.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.124.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.124.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.125.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.125.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.126.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.126.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.127.gate_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.127.up_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.0.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.1.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.2.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.3.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.4.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.5.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.6.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.7.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.8.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.9.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.10.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.11.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.12.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.13.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.14.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.15.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.16.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.17.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.18.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.19.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.20.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.21.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.22.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.23.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.24.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.25.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.26.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.27.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.28.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.29.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.30.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.31.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.32.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.33.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.34.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.35.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.36.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.37.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.38.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.39.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.40.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.41.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.42.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.43.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.44.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.45.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.46.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.47.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.48.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.49.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.50.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.51.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.52.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.53.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.54.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.55.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.56.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.57.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.58.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.59.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.60.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.61.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.62.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.63.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.64.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.65.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.66.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.67.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.68.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.69.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.70.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.71.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.72.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.73.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.74.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.75.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.76.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.77.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.78.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.79.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.80.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.81.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.82.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.83.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.84.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.85.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.86.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.87.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.88.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.89.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.90.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.91.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.92.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.93.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.94.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.95.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.96.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.97.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.98.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.99.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.100.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.101.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.102.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.103.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.104.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.105.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.106.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.107.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.108.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.109.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.110.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.111.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.112.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.113.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.114.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.115.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.116.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.117.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.118.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.119.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.120.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.121.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.122.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.123.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.124.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.125.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.126.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.experts.127.down_proj.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.gate.weight": "model-00016-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.q_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.k_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.v_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.shared_experts.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.shared_experts.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.shared_experts.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.mlp.gate.e_score_correction_bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.o_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.q_proj.bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.k_proj.bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.self_attn.v_proj.bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.post_attention_layernorm.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.18.input_layernorm.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.0.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.0.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.1.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.1.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.2.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.2.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.3.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.3.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.4.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.4.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.5.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.5.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.6.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.6.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.7.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.7.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.8.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.8.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.9.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.9.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.10.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.10.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.11.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.11.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.12.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.12.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.13.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.13.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.14.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.14.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.15.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.15.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.16.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.16.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.17.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.17.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.18.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.18.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.19.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.19.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.20.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.20.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.21.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.21.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.22.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.22.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.23.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.23.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.24.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.24.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.25.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.25.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.26.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.26.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.27.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.27.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.28.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.28.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.29.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.29.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.30.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.30.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.31.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.31.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.32.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.32.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.33.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.33.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.34.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.34.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.35.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.35.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.36.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.36.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.37.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.37.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.38.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.38.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.39.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.39.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.40.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.40.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.41.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.41.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.42.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.42.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.43.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.43.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.44.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.44.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.45.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.45.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.46.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.46.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.47.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.47.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.48.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.48.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.49.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.49.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.50.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.50.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.51.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.51.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.52.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.52.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.53.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.53.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.54.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.54.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.55.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.55.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.56.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.56.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.57.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.57.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.58.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.58.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.59.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.59.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.60.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.60.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.61.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.61.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.62.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.62.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.63.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.63.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.64.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.64.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.65.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.65.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.66.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.66.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.67.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.67.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.68.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.68.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.69.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.69.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.70.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.70.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.71.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.71.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.72.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.72.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.73.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.73.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.74.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.74.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.75.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.75.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.76.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.76.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.77.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.77.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.78.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.78.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.79.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.79.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.80.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.80.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.81.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.81.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.82.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.82.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.83.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.83.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.84.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.84.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.85.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.85.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.86.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.86.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.87.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.87.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.88.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.88.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.89.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.89.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.90.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.90.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.91.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.91.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.92.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.92.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.93.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.93.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.94.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.94.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.95.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.95.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.96.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.96.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.97.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.97.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.98.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.98.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.99.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.99.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.100.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.100.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.101.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.101.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.102.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.102.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.103.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.103.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.104.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.104.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.105.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.105.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.106.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.106.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.107.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.107.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.108.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.108.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.109.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.109.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.110.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.110.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.111.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.111.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.112.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.112.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.113.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.113.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.114.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.114.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.115.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.115.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.116.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.116.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.117.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.117.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.118.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.118.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.119.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.119.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.120.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.120.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.121.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.121.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.122.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.122.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.123.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.123.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.124.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.124.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.125.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.125.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.126.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.126.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.127.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.127.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.0.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.1.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.2.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.3.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.4.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.5.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.6.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.7.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.8.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.9.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.10.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.11.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.12.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.13.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.14.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.15.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.16.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.17.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.18.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.19.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.20.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.21.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.22.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.23.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.24.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.25.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.26.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.27.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.28.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.29.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.30.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.31.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.32.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.33.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.34.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.35.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.36.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.37.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.38.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.39.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.40.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.41.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.42.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.43.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.44.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.45.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.46.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.47.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.48.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.49.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.50.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.51.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.52.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.53.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.54.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.55.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.56.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.57.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.58.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.59.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.60.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.61.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.62.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.63.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.64.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.65.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.66.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.67.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.68.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.69.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.70.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.71.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.72.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.73.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.74.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.75.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.76.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.77.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.78.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.79.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.80.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.81.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.82.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.83.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.84.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.85.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.86.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.87.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.88.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.89.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.90.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.91.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.92.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.93.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.94.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.95.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.96.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.97.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.98.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.99.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.100.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.101.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.102.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.103.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.104.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.105.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.106.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.107.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.108.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.109.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.110.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.111.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.112.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.113.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.114.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.115.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.116.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.117.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.118.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.119.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.120.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.121.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.122.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.123.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.124.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.125.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.126.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.experts.127.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.shared_experts.down_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.gate.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.post_attention_layernorm.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.q_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.k_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.v_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.shared_experts.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.shared_experts.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.input_layernorm.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.mlp.gate.e_score_correction_bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.o_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.q_proj.bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.k_proj.bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.19.self_attn.v_proj.bias": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.0.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.0.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.1.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.1.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.2.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.2.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.3.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.3.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.4.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.4.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.5.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.5.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.6.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.6.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.7.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.7.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.8.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.8.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.9.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.9.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.10.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.10.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.11.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.11.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.12.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.12.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.13.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.13.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.14.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.14.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.15.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.15.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.16.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.16.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.17.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.17.up_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.18.gate_proj.weight": "model-00017-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.18.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.19.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.19.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.20.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.20.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.21.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.21.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.22.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.22.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.23.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.23.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.24.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.24.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.25.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.25.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.26.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.26.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.27.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.27.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.28.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.28.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.29.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.29.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.30.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.30.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.31.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.31.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.32.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.32.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.33.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.33.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.34.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.34.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.35.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.35.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.36.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.36.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.37.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.37.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.38.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.38.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.39.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.39.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.40.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.40.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.41.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.41.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.42.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.42.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.43.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.43.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.44.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.44.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.45.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.45.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.46.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.46.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.47.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.47.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.48.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.48.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.49.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.49.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.50.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.50.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.51.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.51.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.52.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.52.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.53.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.53.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.54.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.54.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.55.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.55.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.56.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.56.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.57.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.57.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.58.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.58.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.59.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.59.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.60.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.60.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.61.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.61.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.62.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.62.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.63.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.63.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.64.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.64.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.65.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.65.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.66.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.66.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.67.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.67.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.68.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.68.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.69.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.69.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.70.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.70.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.71.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.71.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.72.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.72.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.73.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.73.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.74.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.74.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.75.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.75.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.76.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.76.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.77.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.77.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.78.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.78.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.79.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.79.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.80.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.80.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.81.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.81.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.82.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.82.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.83.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.83.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.84.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.84.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.85.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.85.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.86.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.86.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.87.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.87.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.88.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.88.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.89.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.89.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.90.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.90.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.91.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.91.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.92.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.92.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.93.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.93.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.94.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.94.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.95.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.95.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.96.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.96.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.97.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.97.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.98.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.98.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.99.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.99.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.100.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.100.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.101.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.101.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.102.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.102.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.103.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.103.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.104.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.104.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.105.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.105.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.106.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.106.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.107.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.107.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.108.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.108.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.109.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.109.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.110.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.110.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.111.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.111.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.112.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.112.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.113.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.113.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.114.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.114.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.115.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.115.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.116.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.116.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.117.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.117.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.118.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.118.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.119.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.119.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.120.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.120.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.121.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.121.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.122.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.122.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.123.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.123.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.124.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.124.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.125.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.125.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.126.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.126.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.127.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.127.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.0.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.1.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.2.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.3.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.4.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.5.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.6.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.7.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.8.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.9.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.10.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.11.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.12.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.13.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.14.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.15.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.16.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.17.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.18.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.19.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.20.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.21.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.22.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.23.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.24.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.25.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.26.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.27.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.28.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.29.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.30.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.31.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.32.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.33.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.34.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.35.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.36.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.37.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.38.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.39.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.40.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.41.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.42.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.43.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.44.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.45.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.46.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.47.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.48.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.49.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.50.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.51.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.52.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.53.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.54.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.55.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.56.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.57.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.58.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.59.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.60.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.61.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.62.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.63.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.64.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.65.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.66.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.67.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.68.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.69.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.70.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.71.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.72.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.73.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.74.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.75.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.76.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.77.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.78.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.79.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.80.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.81.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.82.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.83.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.84.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.85.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.86.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.87.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.88.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.89.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.90.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.91.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.92.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.93.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.94.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.95.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.96.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.97.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.98.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.99.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.100.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.101.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.102.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.103.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.104.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.105.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.106.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.107.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.108.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.109.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.110.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.111.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.112.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.113.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.114.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.115.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.116.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.117.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.118.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.119.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.120.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.121.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.122.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.123.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.124.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.125.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.126.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.experts.127.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.post_attention_layernorm.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.gate.e_score_correction_bias": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.gate.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.q_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.k_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.v_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.shared_experts.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.shared_experts.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.mlp.shared_experts.down_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.input_layernorm.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.q_proj.bias": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.k_proj.bias": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.v_proj.bias": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.20.self_attn.o_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.0.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.0.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.1.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.1.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.2.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.2.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.3.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.3.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.4.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.4.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.5.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.5.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.6.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.6.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.7.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.7.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.8.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.8.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.9.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.9.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.10.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.10.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.11.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.11.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.12.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.12.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.13.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.13.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.14.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.14.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.15.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.15.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.16.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.16.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.17.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.17.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.18.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.18.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.19.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.19.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.20.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.20.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.21.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.21.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.22.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.22.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.23.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.23.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.24.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.24.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.25.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.25.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.26.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.26.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.27.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.27.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.28.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.28.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.29.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.29.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.30.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.30.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.31.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.31.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.32.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.32.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.33.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.33.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.34.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.34.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.35.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.35.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.36.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.36.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.37.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.37.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.38.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.38.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.39.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.39.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.40.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.40.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.41.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.41.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.42.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.42.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.43.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.43.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.44.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.44.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.45.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.45.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.46.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.46.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.47.gate_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.47.up_proj.weight": "model-00018-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.48.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.48.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.49.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.49.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.50.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.50.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.51.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.51.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.52.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.52.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.53.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.53.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.54.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.54.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.55.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.55.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.56.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.56.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.57.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.57.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.58.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.58.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.59.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.59.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.60.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.60.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.61.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.61.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.62.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.62.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.63.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.63.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.64.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.64.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.65.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.65.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.66.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.66.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.67.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.67.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.68.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.68.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.69.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.69.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.70.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.70.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.71.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.71.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.72.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.72.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.73.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.73.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.74.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.74.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.75.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.75.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.76.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.76.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.77.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.77.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.78.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.78.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.79.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.79.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.80.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.80.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.81.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.81.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.82.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.82.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.83.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.83.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.84.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.84.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.85.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.85.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.86.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.86.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.87.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.87.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.88.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.88.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.89.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.89.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.90.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.90.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.91.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.91.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.92.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.92.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.93.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.93.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.94.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.94.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.95.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.95.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.96.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.96.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.97.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.97.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.98.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.98.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.99.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.99.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.100.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.100.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.101.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.101.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.102.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.102.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.103.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.103.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.104.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.104.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.105.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.105.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.106.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.106.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.107.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.107.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.108.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.108.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.109.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.109.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.110.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.110.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.111.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.111.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.112.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.112.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.113.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.113.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.114.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.114.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.115.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.115.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.116.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.116.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.117.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.117.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.118.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.118.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.119.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.119.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.120.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.120.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.121.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.121.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.122.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.122.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.123.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.123.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.124.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.124.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.125.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.125.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.126.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.126.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.127.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.127.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.0.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.1.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.2.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.3.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.4.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.5.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.6.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.7.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.8.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.9.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.10.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.11.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.12.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.13.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.14.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.15.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.16.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.17.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.18.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.19.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.20.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.21.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.22.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.23.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.24.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.25.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.26.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.27.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.28.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.29.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.30.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.31.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.32.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.33.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.34.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.35.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.36.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.37.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.38.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.39.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.40.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.41.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.42.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.43.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.44.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.45.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.46.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.47.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.48.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.49.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.50.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.51.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.52.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.53.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.54.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.55.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.56.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.57.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.58.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.59.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.60.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.61.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.62.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.63.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.64.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.65.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.66.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.67.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.68.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.69.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.70.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.71.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.72.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.73.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.74.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.75.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.76.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.77.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.78.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.79.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.80.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.81.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.82.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.83.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.84.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.85.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.86.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.87.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.88.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.89.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.90.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.91.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.92.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.93.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.94.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.95.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.96.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.97.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.98.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.99.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.100.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.101.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.102.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.103.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.104.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.105.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.106.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.107.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.108.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.109.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.110.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.111.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.112.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.113.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.114.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.115.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.116.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.117.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.118.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.119.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.120.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.121.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.122.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.123.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.124.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.125.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.126.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.experts.127.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.gate.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.q_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.k_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.v_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.o_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.shared_experts.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.shared_experts.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.shared_experts.down_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.mlp.gate.e_score_correction_bias": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.post_attention_layernorm.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.input_layernorm.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.q_proj.bias": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.k_proj.bias": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.21.self_attn.v_proj.bias": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.0.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.0.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.1.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.1.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.2.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.2.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.3.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.3.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.4.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.4.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.5.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.5.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.6.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.6.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.7.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.7.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.8.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.8.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.9.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.9.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.10.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.10.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.11.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.11.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.12.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.12.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.13.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.13.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.14.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.14.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.15.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.15.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.16.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.16.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.17.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.17.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.18.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.18.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.19.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.19.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.20.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.20.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.21.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.21.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.22.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.22.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.23.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.23.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.24.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.24.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.25.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.25.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.26.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.26.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.27.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.27.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.28.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.28.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.29.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.29.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.30.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.30.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.31.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.31.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.32.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.32.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.33.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.33.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.34.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.34.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.35.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.35.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.36.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.36.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.37.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.37.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.38.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.38.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.39.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.39.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.40.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.40.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.41.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.41.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.42.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.42.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.43.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.43.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.44.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.44.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.45.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.45.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.46.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.46.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.47.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.47.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.48.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.48.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.49.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.49.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.50.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.50.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.51.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.51.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.52.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.52.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.53.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.53.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.54.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.54.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.55.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.55.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.56.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.56.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.57.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.57.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.58.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.58.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.59.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.59.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.60.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.60.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.61.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.61.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.62.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.62.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.63.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.63.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.64.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.64.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.65.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.65.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.66.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.66.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.67.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.67.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.68.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.68.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.69.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.69.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.70.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.70.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.71.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.71.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.72.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.72.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.73.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.73.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.74.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.74.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.75.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.75.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.76.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.76.up_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.77.gate_proj.weight": "model-00019-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.77.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.78.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.78.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.79.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.79.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.80.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.80.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.81.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.81.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.82.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.82.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.83.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.83.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.84.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.84.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.85.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.85.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.86.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.86.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.87.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.87.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.88.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.88.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.89.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.89.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.90.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.90.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.91.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.91.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.92.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.92.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.93.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.93.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.94.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.94.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.95.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.95.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.96.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.96.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.97.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.97.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.98.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.98.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.99.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.99.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.100.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.100.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.101.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.101.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.102.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.102.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.103.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.103.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.104.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.104.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.105.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.105.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.106.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.106.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.107.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.107.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.108.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.108.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.109.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.109.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.110.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.110.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.111.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.111.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.112.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.112.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.113.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.113.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.114.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.114.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.115.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.115.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.116.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.116.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.117.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.117.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.118.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.118.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.119.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.119.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.120.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.120.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.121.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.121.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.122.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.122.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.123.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.123.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.124.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.124.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.125.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.125.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.126.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.126.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.127.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.127.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.0.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.1.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.2.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.3.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.4.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.5.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.6.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.7.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.8.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.9.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.10.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.11.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.12.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.13.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.14.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.15.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.16.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.17.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.18.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.19.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.20.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.21.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.22.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.23.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.24.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.25.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.26.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.27.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.28.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.29.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.30.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.31.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.32.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.33.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.34.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.35.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.36.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.37.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.38.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.39.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.40.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.41.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.42.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.43.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.44.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.45.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.46.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.47.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.48.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.49.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.50.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.51.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.52.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.53.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.54.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.55.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.56.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.57.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.58.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.59.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.60.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.61.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.62.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.63.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.64.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.65.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.66.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.67.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.68.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.69.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.70.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.71.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.72.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.73.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.74.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.75.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.76.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.77.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.78.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.79.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.80.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.81.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.82.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.83.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.84.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.85.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.86.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.87.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.88.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.89.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.90.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.91.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.92.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.93.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.94.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.95.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.96.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.97.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.98.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.99.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.100.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.101.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.102.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.103.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.104.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.105.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.106.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.107.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.108.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.109.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.110.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.111.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.112.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.113.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.114.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.115.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.116.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.117.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.118.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.119.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.120.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.121.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.122.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.123.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.124.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.125.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.126.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.experts.127.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.post_attention_layernorm.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.gate.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.q_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.k_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.v_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.o_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.shared_experts.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.shared_experts.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.input_layernorm.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.q_proj.bias": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.k_proj.bias": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.self_attn.v_proj.bias": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.gate.e_score_correction_bias": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.22.mlp.shared_experts.down_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.0.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.0.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.1.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.1.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.2.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.2.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.3.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.3.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.4.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.4.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.5.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.5.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.6.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.6.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.7.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.7.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.8.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.8.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.9.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.9.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.10.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.10.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.11.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.11.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.12.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.12.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.13.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.13.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.14.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.14.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.15.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.15.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.16.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.16.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.17.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.17.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.18.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.18.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.19.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.19.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.20.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.20.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.21.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.21.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.22.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.22.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.23.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.23.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.24.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.24.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.25.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.25.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.26.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.26.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.27.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.27.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.28.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.28.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.29.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.29.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.30.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.30.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.31.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.31.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.32.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.32.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.33.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.33.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.34.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.34.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.35.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.35.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.36.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.36.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.37.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.37.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.38.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.38.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.39.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.39.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.40.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.40.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.41.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.41.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.42.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.42.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.43.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.43.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.44.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.44.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.45.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.45.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.46.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.46.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.47.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.47.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.48.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.48.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.49.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.49.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.50.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.50.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.51.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.51.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.52.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.52.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.53.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.53.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.54.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.54.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.55.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.55.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.56.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.56.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.57.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.57.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.58.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.58.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.59.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.59.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.60.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.60.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.61.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.61.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.62.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.62.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.63.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.63.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.64.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.64.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.65.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.65.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.66.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.66.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.67.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.67.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.68.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.68.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.69.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.69.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.70.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.70.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.71.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.71.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.72.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.72.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.73.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.73.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.74.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.74.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.75.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.75.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.76.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.76.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.77.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.77.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.78.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.78.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.79.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.79.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.80.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.80.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.81.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.81.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.82.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.82.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.83.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.83.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.84.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.84.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.85.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.85.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.86.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.86.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.87.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.87.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.88.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.88.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.89.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.89.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.90.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.90.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.91.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.91.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.92.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.92.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.93.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.93.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.94.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.94.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.95.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.95.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.96.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.96.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.97.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.97.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.98.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.98.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.99.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.99.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.100.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.100.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.101.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.101.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.102.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.102.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.103.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.103.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.104.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.104.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.105.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.105.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.106.gate_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.106.up_proj.weight": "model-00020-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.107.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.107.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.108.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.108.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.109.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.109.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.110.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.110.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.111.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.111.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.112.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.112.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.113.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.113.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.114.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.114.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.115.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.115.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.116.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.116.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.117.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.117.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.118.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.118.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.119.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.119.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.120.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.120.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.121.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.121.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.122.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.122.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.123.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.123.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.124.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.124.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.125.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.125.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.126.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.126.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.127.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.127.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.0.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.1.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.2.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.3.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.4.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.5.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.6.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.7.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.8.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.9.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.10.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.11.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.12.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.13.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.14.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.15.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.16.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.17.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.18.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.19.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.20.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.21.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.22.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.23.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.24.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.25.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.26.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.27.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.28.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.29.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.30.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.31.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.32.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.33.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.34.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.35.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.36.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.37.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.38.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.39.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.40.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.41.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.42.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.43.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.44.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.45.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.46.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.47.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.48.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.49.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.50.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.51.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.52.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.53.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.54.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.55.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.56.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.57.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.58.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.59.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.60.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.61.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.62.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.63.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.64.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.65.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.66.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.67.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.68.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.69.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.70.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.71.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.72.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.73.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.74.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.75.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.76.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.77.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.78.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.79.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.80.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.81.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.82.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.83.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.84.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.85.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.86.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.87.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.88.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.89.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.90.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.91.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.92.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.93.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.94.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.95.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.96.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.97.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.98.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.99.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.100.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.101.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.102.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.103.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.104.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.105.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.106.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.107.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.108.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.109.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.110.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.111.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.112.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.113.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.114.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.115.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.116.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.117.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.118.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.119.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.120.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.121.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.122.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.123.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.124.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.125.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.126.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.experts.127.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.shared_experts.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.gate.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.q_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.k_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.v_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.o_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.q_proj.bias": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.k_proj.bias": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.self_attn.v_proj.bias": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.post_attention_layernorm.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.shared_experts.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.shared_experts.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.input_layernorm.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.23.mlp.gate.e_score_correction_bias": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.0.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.0.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.1.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.1.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.2.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.2.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.3.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.3.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.4.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.4.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.5.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.5.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.6.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.6.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.7.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.7.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.8.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.8.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.9.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.9.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.10.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.10.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.11.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.11.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.12.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.12.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.13.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.13.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.14.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.14.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.15.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.15.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.16.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.16.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.17.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.17.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.18.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.18.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.19.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.19.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.20.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.20.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.21.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.21.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.22.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.22.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.23.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.23.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.24.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.24.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.25.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.25.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.26.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.26.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.27.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.27.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.28.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.28.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.29.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.29.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.30.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.30.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.31.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.31.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.32.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.32.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.33.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.33.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.34.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.34.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.35.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.35.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.36.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.36.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.37.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.37.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.38.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.38.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.39.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.39.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.40.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.40.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.41.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.41.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.42.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.42.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.43.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.43.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.44.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.44.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.45.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.45.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.46.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.46.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.47.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.47.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.48.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.48.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.49.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.49.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.50.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.50.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.51.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.51.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.52.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.52.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.53.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.53.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.54.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.54.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.55.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.55.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.56.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.56.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.57.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.57.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.58.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.58.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.59.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.59.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.60.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.60.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.61.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.61.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.62.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.62.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.63.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.63.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.64.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.64.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.65.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.65.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.66.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.66.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.67.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.67.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.68.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.68.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.69.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.69.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.70.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.70.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.71.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.71.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.72.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.72.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.73.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.73.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.74.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.74.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.75.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.75.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.76.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.76.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.77.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.77.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.78.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.78.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.79.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.79.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.80.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.80.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.81.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.81.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.82.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.82.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.83.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.83.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.84.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.84.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.85.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.85.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.86.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.86.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.87.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.87.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.88.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.88.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.89.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.89.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.90.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.90.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.91.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.91.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.92.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.92.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.93.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.93.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.94.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.94.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.95.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.95.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.96.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.96.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.97.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.97.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.98.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.98.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.99.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.99.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.100.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.100.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.101.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.101.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.102.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.102.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.103.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.103.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.104.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.104.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.105.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.105.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.106.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.106.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.107.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.107.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.108.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.108.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.109.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.109.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.110.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.110.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.111.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.111.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.112.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.112.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.113.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.113.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.114.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.114.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.115.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.115.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.116.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.116.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.117.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.117.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.118.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.118.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.119.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.119.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.120.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.120.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.121.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.121.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.122.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.122.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.123.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.123.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.124.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.124.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.125.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.125.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.126.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.126.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.127.gate_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.127.up_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.0.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.1.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.2.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.3.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.4.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.5.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.6.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.7.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.8.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.9.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.10.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.11.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.12.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.13.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.14.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.15.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.16.down_proj.weight": "model-00021-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.17.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.18.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.19.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.20.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.21.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.22.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.23.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.24.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.25.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.26.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.27.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.28.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.29.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.30.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.31.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.32.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.33.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.34.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.35.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.36.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.37.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.38.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.39.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.40.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.41.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.42.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.43.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.44.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.45.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.46.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.47.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.48.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.49.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.50.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.51.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.52.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.53.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.54.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.55.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.56.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.57.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.58.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.59.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.60.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.61.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.62.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.63.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.64.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.65.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.66.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.67.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.68.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.69.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.70.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.71.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.72.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.73.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.74.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.75.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.76.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.77.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.78.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.79.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.80.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.81.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.82.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.83.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.84.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.85.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.86.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.87.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.88.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.89.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.90.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.91.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.92.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.93.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.94.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.95.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.96.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.97.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.98.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.99.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.100.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.101.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.102.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.103.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.104.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.105.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.106.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.107.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.108.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.109.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.110.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.111.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.112.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.113.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.114.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.115.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.116.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.117.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.118.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.119.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.120.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.121.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.122.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.123.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.124.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.125.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.126.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.experts.127.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.shared_experts.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.shared_experts.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.post_attention_layernorm.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.gate.e_score_correction_bias": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.shared_experts.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.mlp.gate.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.q_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.k_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.v_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.o_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.input_layernorm.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.q_proj.bias": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.k_proj.bias": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.24.self_attn.v_proj.bias": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.0.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.0.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.1.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.1.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.2.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.2.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.3.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.3.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.4.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.4.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.5.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.5.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.6.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.6.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.7.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.7.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.8.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.8.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.9.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.9.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.10.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.10.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.11.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.11.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.12.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.12.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.13.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.13.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.14.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.14.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.15.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.15.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.16.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.16.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.17.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.17.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.18.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.18.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.19.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.19.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.20.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.20.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.21.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.21.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.22.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.22.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.23.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.23.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.24.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.24.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.25.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.25.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.26.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.26.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.27.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.27.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.28.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.28.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.29.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.29.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.30.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.30.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.31.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.31.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.32.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.32.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.33.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.33.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.34.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.34.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.35.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.35.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.36.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.36.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.37.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.37.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.38.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.38.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.39.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.39.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.40.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.40.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.41.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.41.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.42.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.42.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.43.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.43.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.44.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.44.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.45.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.45.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.46.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.46.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.47.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.47.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.48.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.48.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.49.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.49.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.50.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.50.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.51.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.51.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.52.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.52.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.53.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.53.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.54.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.54.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.55.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.55.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.56.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.56.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.57.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.57.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.58.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.58.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.59.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.59.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.60.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.60.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.61.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.61.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.62.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.62.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.63.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.63.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.64.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.64.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.65.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.65.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.66.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.66.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.67.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.67.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.68.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.68.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.69.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.69.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.70.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.70.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.71.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.71.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.72.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.72.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.73.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.73.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.74.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.74.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.75.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.75.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.76.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.76.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.77.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.77.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.78.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.78.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.79.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.79.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.80.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.80.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.81.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.81.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.82.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.82.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.83.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.83.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.84.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.84.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.85.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.85.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.86.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.86.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.87.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.87.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.88.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.88.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.89.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.89.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.90.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.90.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.91.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.91.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.92.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.92.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.93.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.93.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.94.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.94.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.95.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.95.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.96.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.96.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.97.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.97.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.98.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.98.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.99.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.99.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.100.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.100.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.101.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.101.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.102.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.102.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.103.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.103.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.104.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.104.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.105.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.105.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.106.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.106.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.107.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.107.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.108.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.108.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.109.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.109.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.110.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.110.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.111.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.111.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.112.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.112.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.113.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.113.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.114.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.114.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.115.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.115.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.116.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.116.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.117.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.117.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.118.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.118.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.119.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.119.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.120.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.120.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.121.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.121.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.122.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.122.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.123.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.123.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.124.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.124.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.125.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.125.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.126.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.126.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.127.gate_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.127.up_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.0.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.1.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.2.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.3.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.4.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.5.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.6.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.7.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.8.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.9.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.10.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.11.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.12.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.13.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.14.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.15.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.16.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.17.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.18.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.19.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.20.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.21.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.22.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.23.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.24.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.25.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.26.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.27.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.28.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.29.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.30.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.31.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.32.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.33.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.34.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.35.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.36.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.37.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.38.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.39.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.40.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.41.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.42.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.43.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.44.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.45.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.46.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.47.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.48.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.49.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.50.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.51.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.52.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.53.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.54.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.55.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.56.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.57.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.58.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.59.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.60.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.61.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.62.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.63.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.64.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.65.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.66.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.67.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.68.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.69.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.70.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.71.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.72.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.73.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.74.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.75.down_proj.weight": "model-00022-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.76.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.77.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.78.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.79.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.80.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.81.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.82.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.83.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.84.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.85.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.86.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.87.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.88.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.89.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.90.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.91.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.92.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.93.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.94.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.95.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.96.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.97.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.98.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.99.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.100.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.101.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.102.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.103.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.104.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.105.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.106.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.107.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.108.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.109.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.110.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.111.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.112.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.113.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.114.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.115.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.116.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.117.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.118.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.119.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.120.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.121.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.122.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.123.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.124.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.125.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.126.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.experts.127.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.shared_experts.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.shared_experts.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.shared_experts.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.gate.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.q_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.k_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.v_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.o_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.q_proj.bias": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.k_proj.bias": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.self_attn.v_proj.bias": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.mlp.gate.e_score_correction_bias": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.post_attention_layernorm.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.25.input_layernorm.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.0.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.0.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.1.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.1.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.2.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.2.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.3.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.3.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.4.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.4.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.5.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.5.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.6.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.6.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.7.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.7.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.8.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.8.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.9.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.9.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.10.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.10.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.11.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.11.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.12.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.12.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.13.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.13.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.14.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.14.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.15.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.15.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.16.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.16.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.17.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.17.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.18.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.18.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.19.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.19.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.20.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.20.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.21.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.21.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.22.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.22.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.23.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.23.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.24.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.24.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.25.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.25.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.26.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.26.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.27.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.27.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.28.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.28.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.29.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.29.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.30.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.30.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.31.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.31.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.32.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.32.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.33.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.33.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.34.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.34.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.35.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.35.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.36.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.36.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.37.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.37.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.38.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.38.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.39.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.39.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.40.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.40.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.41.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.41.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.42.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.42.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.43.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.43.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.44.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.44.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.45.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.45.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.46.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.46.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.47.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.47.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.48.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.48.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.49.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.49.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.50.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.50.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.51.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.51.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.52.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.52.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.53.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.53.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.54.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.54.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.55.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.55.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.56.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.56.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.57.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.57.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.58.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.58.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.59.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.59.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.60.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.60.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.61.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.61.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.62.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.62.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.63.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.63.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.64.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.64.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.65.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.65.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.66.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.66.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.67.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.67.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.68.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.68.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.69.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.69.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.70.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.70.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.71.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.71.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.72.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.72.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.73.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.73.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.74.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.74.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.75.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.75.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.76.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.76.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.77.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.77.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.78.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.78.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.79.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.79.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.80.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.80.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.81.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.81.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.82.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.82.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.83.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.83.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.84.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.84.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.85.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.85.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.86.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.86.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.87.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.87.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.88.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.88.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.89.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.89.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.90.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.90.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.91.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.91.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.92.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.92.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.93.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.93.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.94.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.94.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.95.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.95.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.96.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.96.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.97.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.97.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.98.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.98.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.99.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.99.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.100.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.100.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.101.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.101.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.102.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.102.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.103.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.103.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.104.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.104.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.105.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.105.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.106.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.106.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.107.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.107.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.108.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.108.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.109.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.109.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.110.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.110.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.111.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.111.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.112.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.112.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.113.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.113.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.114.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.114.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.115.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.115.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.116.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.116.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.117.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.117.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.118.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.118.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.119.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.119.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.120.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.120.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.121.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.121.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.122.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.122.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.123.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.123.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.124.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.124.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.125.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.125.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.126.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.126.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.127.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.127.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.0.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.1.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.2.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.3.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.4.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.5.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.6.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.7.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.8.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.9.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.10.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.11.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.12.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.13.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.14.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.15.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.16.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.17.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.18.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.19.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.20.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.21.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.22.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.23.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.24.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.25.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.26.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.27.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.28.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.29.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.30.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.31.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.32.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.33.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.34.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.35.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.36.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.37.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.38.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.39.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.40.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.41.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.42.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.43.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.44.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.45.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.46.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.47.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.48.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.49.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.50.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.51.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.52.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.53.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.54.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.55.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.56.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.57.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.58.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.59.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.60.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.61.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.62.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.63.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.64.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.65.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.66.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.67.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.68.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.69.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.70.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.71.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.72.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.73.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.74.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.75.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.76.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.77.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.78.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.79.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.80.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.81.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.82.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.83.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.84.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.85.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.86.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.87.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.88.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.89.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.90.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.91.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.92.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.93.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.94.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.95.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.96.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.97.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.98.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.99.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.100.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.101.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.102.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.103.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.104.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.105.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.106.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.107.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.108.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.109.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.110.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.111.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.112.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.113.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.114.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.115.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.116.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.117.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.118.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.119.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.120.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.121.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.122.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.123.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.124.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.125.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.126.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.experts.127.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.post_attention_layernorm.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.shared_experts.gate_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.shared_experts.up_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.shared_experts.down_proj.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.gate.weight": "model-00023-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.q_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.k_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.v_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.o_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.input_layernorm.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.mlp.gate.e_score_correction_bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.q_proj.bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.k_proj.bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.26.self_attn.v_proj.bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.0.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.0.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.1.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.1.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.2.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.2.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.3.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.3.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.4.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.4.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.5.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.5.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.6.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.6.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.7.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.7.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.8.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.8.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.9.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.9.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.10.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.10.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.11.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.11.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.12.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.12.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.13.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.13.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.14.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.14.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.15.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.15.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.16.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.16.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.17.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.17.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.18.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.18.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.19.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.19.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.20.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.20.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.21.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.21.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.22.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.22.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.23.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.23.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.24.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.24.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.25.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.25.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.26.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.26.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.27.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.27.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.28.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.28.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.29.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.29.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.30.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.30.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.31.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.31.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.32.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.32.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.33.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.33.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.34.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.34.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.35.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.35.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.36.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.36.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.37.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.37.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.38.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.38.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.39.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.39.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.40.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.40.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.41.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.41.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.42.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.42.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.43.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.43.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.44.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.44.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.45.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.45.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.46.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.46.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.47.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.47.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.48.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.48.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.49.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.49.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.50.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.50.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.51.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.51.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.52.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.52.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.53.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.53.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.54.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.54.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.55.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.55.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.56.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.56.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.57.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.57.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.58.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.58.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.59.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.59.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.60.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.60.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.61.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.61.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.62.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.62.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.63.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.63.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.64.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.64.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.65.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.65.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.66.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.66.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.67.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.67.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.68.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.68.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.69.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.69.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.70.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.70.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.71.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.71.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.72.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.72.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.73.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.73.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.74.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.74.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.75.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.75.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.76.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.76.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.77.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.77.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.78.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.78.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.79.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.79.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.80.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.80.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.81.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.81.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.82.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.82.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.83.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.83.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.84.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.84.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.85.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.85.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.86.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.86.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.87.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.87.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.88.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.88.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.89.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.89.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.90.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.90.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.91.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.91.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.92.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.92.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.93.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.93.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.94.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.94.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.95.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.95.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.96.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.96.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.97.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.97.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.98.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.98.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.99.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.99.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.100.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.100.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.101.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.101.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.102.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.102.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.103.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.103.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.104.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.104.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.105.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.105.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.106.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.106.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.107.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.107.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.108.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.108.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.109.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.109.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.110.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.110.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.111.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.111.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.112.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.112.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.113.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.113.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.114.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.114.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.115.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.115.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.116.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.116.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.117.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.117.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.118.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.118.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.119.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.119.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.120.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.120.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.121.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.121.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.122.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.122.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.123.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.123.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.124.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.124.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.125.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.125.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.126.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.126.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.127.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.127.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.0.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.1.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.2.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.3.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.4.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.5.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.6.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.7.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.8.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.9.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.10.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.11.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.12.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.13.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.14.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.15.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.16.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.17.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.18.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.19.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.20.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.21.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.22.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.23.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.24.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.25.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.26.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.27.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.28.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.29.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.30.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.31.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.32.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.33.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.34.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.35.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.36.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.37.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.38.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.39.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.40.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.41.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.42.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.43.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.44.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.45.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.46.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.47.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.48.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.49.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.50.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.51.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.52.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.53.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.54.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.55.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.56.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.57.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.58.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.59.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.60.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.61.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.62.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.63.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.64.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.65.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.66.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.67.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.68.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.69.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.70.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.71.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.72.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.73.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.74.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.75.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.76.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.77.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.78.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.79.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.80.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.81.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.82.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.83.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.84.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.85.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.86.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.87.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.88.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.89.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.90.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.91.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.92.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.93.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.94.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.95.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.96.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.97.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.98.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.99.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.100.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.101.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.102.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.103.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.104.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.105.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.106.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.107.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.108.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.109.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.110.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.111.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.112.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.113.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.114.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.115.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.116.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.117.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.118.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.119.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.120.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.121.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.122.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.123.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.124.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.125.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.126.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.experts.127.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.gate.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.shared_experts.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.shared_experts.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.shared_experts.down_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.q_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.k_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.v_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.o_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.post_attention_layernorm.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.input_layernorm.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.mlp.gate.e_score_correction_bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.q_proj.bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.k_proj.bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.27.self_attn.v_proj.bias": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.0.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.0.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.1.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.1.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.2.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.2.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.3.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.3.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.4.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.4.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.5.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.5.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.6.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.6.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.7.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.7.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.8.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.8.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.9.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.9.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.10.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.10.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.11.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.11.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.12.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.12.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.13.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.13.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.14.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.14.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.15.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.15.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.16.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.16.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.17.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.17.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.18.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.18.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.19.gate_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.19.up_proj.weight": "model-00024-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.20.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.20.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.21.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.21.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.22.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.22.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.23.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.23.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.24.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.24.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.25.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.25.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.26.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.26.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.27.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.27.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.28.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.28.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.29.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.29.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.30.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.30.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.31.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.31.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.32.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.32.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.33.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.33.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.34.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.34.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.35.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.35.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.36.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.36.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.37.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.37.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.38.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.38.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.39.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.39.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.40.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.40.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.41.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.41.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.42.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.42.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.43.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.43.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.44.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.44.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.45.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.45.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.46.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.46.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.47.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.47.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.48.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.48.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.49.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.49.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.50.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.50.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.51.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.51.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.52.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.52.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.53.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.53.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.54.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.54.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.55.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.55.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.56.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.56.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.57.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.57.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.58.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.58.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.59.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.59.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.60.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.60.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.61.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.61.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.62.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.62.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.63.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.63.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.64.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.64.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.65.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.65.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.66.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.66.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.67.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.67.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.68.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.68.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.69.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.69.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.70.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.70.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.71.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.71.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.72.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.72.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.73.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.73.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.74.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.74.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.75.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.75.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.76.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.76.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.77.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.77.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.78.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.78.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.79.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.79.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.80.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.80.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.81.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.81.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.82.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.82.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.83.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.83.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.84.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.84.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.85.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.85.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.86.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.86.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.87.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.87.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.88.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.88.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.89.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.89.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.90.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.90.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.91.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.91.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.92.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.92.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.93.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.93.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.94.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.94.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.95.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.95.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.96.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.96.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.97.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.97.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.98.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.98.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.99.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.99.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.100.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.100.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.101.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.101.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.102.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.102.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.103.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.103.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.104.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.104.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.105.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.105.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.106.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.106.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.107.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.107.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.108.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.108.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.109.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.109.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.110.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.110.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.111.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.111.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.112.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.112.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.113.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.113.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.114.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.114.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.115.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.115.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.116.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.116.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.117.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.117.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.118.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.118.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.119.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.119.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.120.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.120.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.121.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.121.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.122.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.122.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.123.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.123.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.124.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.124.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.125.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.125.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.126.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.126.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.127.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.127.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.0.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.1.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.2.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.3.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.4.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.5.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.6.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.7.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.8.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.9.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.10.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.11.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.12.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.13.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.14.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.15.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.16.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.17.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.18.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.19.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.20.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.21.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.22.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.23.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.24.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.25.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.26.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.27.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.28.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.29.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.30.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.31.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.32.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.33.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.34.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.35.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.36.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.37.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.38.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.39.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.40.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.41.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.42.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.43.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.44.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.45.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.46.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.47.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.48.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.49.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.50.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.51.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.52.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.53.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.54.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.55.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.56.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.57.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.58.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.59.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.60.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.61.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.62.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.63.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.64.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.65.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.66.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.67.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.68.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.69.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.70.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.71.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.72.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.73.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.74.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.75.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.76.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.77.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.78.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.79.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.80.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.81.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.82.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.83.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.84.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.85.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.86.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.87.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.88.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.89.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.90.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.91.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.92.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.93.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.94.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.95.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.96.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.97.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.98.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.99.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.100.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.101.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.102.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.103.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.104.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.105.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.106.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.107.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.108.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.109.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.110.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.111.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.112.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.113.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.114.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.115.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.116.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.117.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.118.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.119.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.120.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.121.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.122.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.123.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.124.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.125.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.126.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.experts.127.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.gate.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.shared_experts.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.shared_experts.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.shared_experts.down_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.input_layernorm.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.q_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.k_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.v_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.o_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.q_proj.bias": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.k_proj.bias": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.self_attn.v_proj.bias": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.post_attention_layernorm.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.28.mlp.gate.e_score_correction_bias": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.0.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.0.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.1.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.1.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.2.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.2.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.3.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.3.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.4.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.4.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.5.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.5.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.6.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.6.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.7.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.7.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.8.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.8.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.9.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.9.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.10.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.10.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.11.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.11.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.12.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.12.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.13.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.13.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.14.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.14.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.15.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.15.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.16.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.16.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.17.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.17.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.18.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.18.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.19.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.19.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.20.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.20.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.21.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.21.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.22.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.22.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.23.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.23.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.24.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.24.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.25.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.25.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.26.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.26.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.27.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.27.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.28.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.28.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.29.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.29.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.30.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.30.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.31.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.31.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.32.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.32.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.33.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.33.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.34.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.34.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.35.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.35.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.36.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.36.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.37.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.37.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.38.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.38.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.39.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.39.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.40.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.40.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.41.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.41.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.42.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.42.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.43.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.43.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.44.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.44.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.45.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.45.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.46.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.46.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.47.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.47.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.48.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.48.up_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.49.gate_proj.weight": "model-00025-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.49.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.50.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.50.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.51.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.51.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.52.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.52.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.53.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.53.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.54.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.54.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.55.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.55.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.56.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.56.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.57.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.57.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.58.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.58.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.59.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.59.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.60.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.60.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.61.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.61.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.62.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.62.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.63.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.63.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.64.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.64.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.65.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.65.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.66.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.66.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.67.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.67.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.68.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.68.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.69.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.69.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.70.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.70.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.71.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.71.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.72.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.72.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.73.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.73.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.74.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.74.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.75.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.75.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.76.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.76.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.77.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.77.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.78.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.78.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.79.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.79.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.80.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.80.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.81.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.81.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.82.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.82.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.83.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.83.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.84.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.84.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.85.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.85.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.86.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.86.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.87.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.87.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.88.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.88.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.89.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.89.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.90.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.90.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.91.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.91.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.92.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.92.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.93.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.93.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.94.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.94.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.95.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.95.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.96.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.96.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.97.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.97.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.98.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.98.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.99.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.99.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.100.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.100.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.101.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.101.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.102.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.102.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.103.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.103.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.104.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.104.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.105.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.105.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.106.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.106.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.107.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.107.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.108.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.108.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.109.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.109.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.110.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.110.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.111.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.111.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.112.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.112.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.113.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.113.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.114.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.114.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.115.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.115.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.116.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.116.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.117.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.117.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.118.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.118.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.119.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.119.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.120.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.120.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.121.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.121.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.122.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.122.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.123.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.123.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.124.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.124.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.125.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.125.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.126.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.126.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.127.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.127.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.0.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.1.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.2.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.3.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.4.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.5.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.6.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.7.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.8.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.9.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.10.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.11.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.12.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.13.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.14.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.15.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.16.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.17.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.18.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.19.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.20.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.21.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.22.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.23.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.24.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.25.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.26.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.27.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.28.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.29.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.30.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.31.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.32.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.33.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.34.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.35.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.36.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.37.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.38.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.39.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.40.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.41.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.42.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.43.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.44.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.45.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.46.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.47.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.48.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.49.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.50.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.51.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.52.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.53.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.54.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.55.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.56.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.57.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.58.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.59.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.60.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.61.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.62.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.63.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.64.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.65.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.66.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.67.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.68.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.69.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.70.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.71.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.72.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.73.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.74.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.75.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.76.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.77.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.78.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.79.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.80.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.81.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.82.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.83.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.84.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.85.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.86.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.87.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.88.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.89.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.90.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.91.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.92.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.93.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.94.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.95.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.96.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.97.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.98.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.99.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.100.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.101.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.102.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.103.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.104.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.105.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.106.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.107.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.108.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.109.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.110.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.111.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.112.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.113.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.114.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.115.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.116.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.117.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.118.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.119.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.120.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.121.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.122.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.123.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.124.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.125.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.126.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.experts.127.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.gate.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.gate.e_score_correction_bias": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.shared_experts.down_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.q_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.k_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.v_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.o_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.post_attention_layernorm.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.shared_experts.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.mlp.shared_experts.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.input_layernorm.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.q_proj.bias": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.k_proj.bias": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.29.self_attn.v_proj.bias": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.0.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.0.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.1.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.1.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.2.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.2.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.3.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.3.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.4.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.4.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.5.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.5.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.6.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.6.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.7.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.7.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.8.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.8.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.9.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.9.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.10.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.10.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.11.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.11.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.12.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.12.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.13.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.13.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.14.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.14.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.15.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.15.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.16.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.16.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.17.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.17.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.18.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.18.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.19.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.19.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.20.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.20.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.21.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.21.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.22.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.22.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.23.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.23.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.24.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.24.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.25.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.25.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.26.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.26.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.27.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.27.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.28.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.28.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.29.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.29.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.30.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.30.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.31.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.31.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.32.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.32.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.33.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.33.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.34.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.34.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.35.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.35.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.36.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.36.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.37.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.37.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.38.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.38.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.39.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.39.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.40.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.40.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.41.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.41.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.42.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.42.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.43.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.43.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.44.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.44.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.45.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.45.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.46.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.46.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.47.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.47.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.48.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.48.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.49.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.49.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.50.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.50.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.51.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.51.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.52.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.52.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.53.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.53.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.54.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.54.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.55.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.55.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.56.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.56.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.57.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.57.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.58.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.58.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.59.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.59.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.60.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.60.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.61.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.61.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.62.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.62.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.63.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.63.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.64.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.64.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.65.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.65.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.66.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.66.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.67.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.67.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.68.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.68.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.69.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.69.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.70.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.70.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.71.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.71.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.72.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.72.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.73.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.73.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.74.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.74.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.75.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.75.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.76.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.76.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.77.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.77.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.78.gate_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.78.up_proj.weight": "model-00026-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.79.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.79.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.80.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.80.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.81.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.81.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.82.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.82.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.83.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.83.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.84.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.84.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.85.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.85.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.86.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.86.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.87.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.87.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.88.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.88.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.89.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.89.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.90.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.90.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.91.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.91.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.92.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.92.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.93.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.93.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.94.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.94.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.95.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.95.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.96.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.96.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.97.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.97.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.98.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.98.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.99.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.99.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.100.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.100.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.101.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.101.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.102.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.102.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.103.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.103.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.104.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.104.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.105.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.105.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.106.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.106.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.107.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.107.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.108.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.108.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.109.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.109.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.110.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.110.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.111.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.111.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.112.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.112.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.113.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.113.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.114.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.114.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.115.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.115.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.116.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.116.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.117.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.117.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.118.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.118.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.119.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.119.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.120.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.120.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.121.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.121.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.122.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.122.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.123.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.123.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.124.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.124.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.125.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.125.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.126.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.126.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.127.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.127.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.0.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.1.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.2.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.3.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.4.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.5.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.6.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.7.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.8.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.9.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.10.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.11.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.12.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.13.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.14.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.15.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.16.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.17.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.18.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.19.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.20.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.21.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.22.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.23.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.24.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.25.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.26.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.27.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.28.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.29.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.30.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.31.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.32.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.33.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.34.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.35.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.36.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.37.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.38.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.39.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.40.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.41.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.42.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.43.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.44.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.45.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.46.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.47.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.48.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.49.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.50.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.51.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.52.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.53.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.54.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.55.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.56.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.57.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.58.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.59.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.60.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.61.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.62.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.63.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.64.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.65.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.66.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.67.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.68.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.69.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.70.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.71.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.72.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.73.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.74.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.75.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.76.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.77.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.78.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.79.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.80.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.81.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.82.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.83.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.84.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.85.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.86.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.87.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.88.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.89.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.90.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.91.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.92.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.93.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.94.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.95.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.96.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.97.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.98.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.99.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.100.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.101.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.102.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.103.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.104.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.105.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.106.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.107.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.108.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.109.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.110.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.111.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.112.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.113.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.114.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.115.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.116.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.117.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.118.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.119.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.120.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.121.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.122.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.123.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.124.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.125.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.126.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.experts.127.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.post_attention_layernorm.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.gate.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.shared_experts.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.shared_experts.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.shared_experts.down_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.input_layernorm.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.q_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.k_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.v_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.q_proj.bias": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.k_proj.bias": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.v_proj.bias": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.mlp.gate.e_score_correction_bias": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.30.self_attn.o_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.0.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.0.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.1.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.1.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.2.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.2.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.3.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.3.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.4.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.4.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.5.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.5.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.6.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.6.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.7.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.7.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.8.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.8.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.9.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.9.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.10.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.10.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.11.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.11.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.12.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.12.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.13.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.13.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.14.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.14.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.15.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.15.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.16.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.16.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.17.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.17.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.18.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.18.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.19.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.19.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.20.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.20.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.21.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.21.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.22.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.22.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.23.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.23.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.24.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.24.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.25.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.25.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.26.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.26.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.27.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.27.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.28.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.28.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.29.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.29.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.30.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.30.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.31.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.31.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.32.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.32.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.33.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.33.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.34.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.34.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.35.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.35.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.36.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.36.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.37.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.37.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.38.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.38.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.39.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.39.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.40.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.40.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.41.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.41.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.42.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.42.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.43.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.43.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.44.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.44.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.45.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.45.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.46.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.46.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.47.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.47.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.48.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.48.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.49.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.49.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.50.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.50.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.51.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.51.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.52.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.52.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.53.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.53.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.54.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.54.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.55.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.55.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.56.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.56.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.57.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.57.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.58.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.58.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.59.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.59.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.60.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.60.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.61.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.61.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.62.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.62.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.63.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.63.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.64.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.64.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.65.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.65.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.66.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.66.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.67.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.67.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.68.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.68.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.69.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.69.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.70.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.70.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.71.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.71.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.72.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.72.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.73.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.73.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.74.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.74.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.75.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.75.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.76.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.76.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.77.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.77.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.78.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.78.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.79.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.79.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.80.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.80.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.81.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.81.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.82.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.82.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.83.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.83.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.84.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.84.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.85.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.85.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.86.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.86.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.87.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.87.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.88.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.88.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.89.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.89.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.90.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.90.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.91.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.91.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.92.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.92.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.93.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.93.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.94.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.94.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.95.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.95.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.96.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.96.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.97.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.97.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.98.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.98.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.99.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.99.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.100.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.100.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.101.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.101.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.102.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.102.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.103.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.103.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.104.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.104.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.105.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.105.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.106.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.106.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.107.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.107.up_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.108.gate_proj.weight": "model-00027-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.108.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.109.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.109.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.110.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.110.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.111.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.111.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.112.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.112.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.113.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.113.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.114.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.114.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.115.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.115.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.116.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.116.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.117.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.117.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.118.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.118.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.119.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.119.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.120.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.120.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.121.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.121.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.122.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.122.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.123.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.123.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.124.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.124.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.125.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.125.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.126.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.126.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.127.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.127.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.0.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.1.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.2.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.3.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.4.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.5.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.6.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.7.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.8.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.9.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.10.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.11.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.12.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.13.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.14.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.15.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.16.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.17.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.18.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.19.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.20.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.21.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.22.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.23.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.24.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.25.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.26.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.27.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.28.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.29.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.30.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.31.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.32.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.33.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.34.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.35.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.36.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.37.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.38.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.39.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.40.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.41.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.42.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.43.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.44.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.45.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.46.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.47.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.48.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.49.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.50.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.51.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.52.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.53.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.54.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.55.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.56.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.57.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.58.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.59.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.60.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.61.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.62.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.63.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.64.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.65.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.66.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.67.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.68.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.69.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.70.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.71.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.72.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.73.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.74.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.75.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.76.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.77.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.78.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.79.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.80.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.81.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.82.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.83.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.84.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.85.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.86.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.87.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.88.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.89.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.90.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.91.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.92.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.93.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.94.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.95.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.96.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.97.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.98.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.99.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.100.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.101.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.102.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.103.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.104.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.105.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.106.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.107.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.108.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.109.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.110.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.111.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.112.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.113.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.114.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.115.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.116.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.117.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.118.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.119.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.120.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.121.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.122.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.123.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.124.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.125.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.126.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.experts.127.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.gate.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.shared_experts.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.shared_experts.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.q_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.k_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.v_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.post_attention_layernorm.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.o_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.shared_experts.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.input_layernorm.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.q_proj.bias": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.k_proj.bias": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.self_attn.v_proj.bias": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.31.mlp.gate.e_score_correction_bias": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.0.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.0.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.1.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.1.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.2.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.2.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.3.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.3.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.4.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.4.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.5.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.5.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.6.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.6.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.7.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.7.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.8.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.8.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.9.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.9.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.10.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.10.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.11.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.11.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.12.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.12.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.13.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.13.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.14.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.14.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.15.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.15.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.16.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.16.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.17.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.17.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.18.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.18.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.19.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.19.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.20.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.20.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.21.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.21.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.22.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.22.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.23.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.23.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.24.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.24.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.25.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.25.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.26.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.26.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.27.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.27.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.28.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.28.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.29.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.29.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.30.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.30.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.31.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.31.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.32.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.32.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.33.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.33.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.34.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.34.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.35.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.35.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.36.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.36.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.37.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.37.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.38.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.38.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.39.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.39.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.40.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.40.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.41.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.41.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.42.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.42.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.43.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.43.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.44.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.44.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.45.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.45.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.46.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.46.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.47.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.47.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.48.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.48.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.49.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.49.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.50.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.50.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.51.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.51.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.52.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.52.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.53.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.53.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.54.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.54.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.55.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.55.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.56.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.56.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.57.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.57.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.58.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.58.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.59.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.59.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.60.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.60.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.61.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.61.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.62.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.62.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.63.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.63.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.64.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.64.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.65.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.65.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.66.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.66.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.67.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.67.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.68.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.68.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.69.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.69.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.70.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.70.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.71.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.71.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.72.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.72.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.73.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.73.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.74.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.74.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.75.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.75.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.76.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.76.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.77.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.77.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.78.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.78.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.79.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.79.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.80.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.80.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.81.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.81.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.82.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.82.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.83.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.83.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.84.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.84.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.85.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.85.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.86.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.86.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.87.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.87.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.88.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.88.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.89.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.89.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.90.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.90.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.91.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.91.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.92.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.92.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.93.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.93.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.94.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.94.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.95.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.95.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.96.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.96.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.97.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.97.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.98.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.98.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.99.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.99.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.100.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.100.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.101.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.101.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.102.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.102.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.103.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.103.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.104.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.104.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.105.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.105.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.106.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.106.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.107.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.107.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.108.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.108.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.109.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.109.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.110.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.110.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.111.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.111.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.112.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.112.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.113.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.113.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.114.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.114.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.115.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.115.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.116.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.116.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.117.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.117.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.118.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.118.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.119.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.119.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.120.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.120.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.121.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.121.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.122.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.122.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.123.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.123.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.124.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.124.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.125.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.125.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.126.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.126.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.127.gate_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.127.up_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.0.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.1.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.2.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.3.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.4.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.5.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.6.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.7.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.8.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.9.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.10.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.11.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.12.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.13.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.14.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.15.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.16.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.17.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.18.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.19.down_proj.weight": "model-00028-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.20.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.21.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.22.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.23.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.24.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.25.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.26.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.27.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.28.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.29.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.30.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.31.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.32.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.33.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.34.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.35.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.36.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.37.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.38.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.39.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.40.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.41.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.42.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.43.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.44.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.45.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.46.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.47.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.48.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.49.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.50.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.51.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.52.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.53.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.54.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.55.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.56.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.57.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.58.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.59.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.60.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.61.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.62.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.63.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.64.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.65.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.66.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.67.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.68.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.69.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.70.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.71.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.72.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.73.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.74.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.75.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.76.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.77.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.78.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.79.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.80.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.81.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.82.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.83.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.84.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.85.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.86.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.87.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.88.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.89.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.90.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.91.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.92.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.93.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.94.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.95.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.96.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.97.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.98.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.99.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.100.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.101.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.102.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.103.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.104.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.105.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.106.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.107.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.108.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.109.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.110.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.111.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.112.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.113.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.114.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.115.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.116.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.117.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.118.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.119.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.120.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.121.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.122.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.123.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.124.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.125.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.126.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.experts.127.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.shared_experts.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.input_layernorm.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.gate.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.q_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.k_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.v_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.o_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.shared_experts.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.shared_experts.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.q_proj.bias": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.k_proj.bias": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.self_attn.v_proj.bias": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.post_attention_layernorm.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.32.mlp.gate.e_score_correction_bias": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.0.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.0.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.1.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.1.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.2.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.2.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.3.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.3.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.4.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.4.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.5.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.5.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.6.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.6.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.7.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.7.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.8.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.8.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.9.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.9.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.10.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.10.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.11.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.11.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.12.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.12.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.13.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.13.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.14.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.14.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.15.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.15.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.16.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.16.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.17.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.17.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.18.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.18.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.19.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.19.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.20.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.20.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.21.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.21.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.22.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.22.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.23.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.23.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.24.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.24.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.25.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.25.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.26.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.26.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.27.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.27.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.28.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.28.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.29.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.29.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.30.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.30.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.31.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.31.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.32.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.32.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.33.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.33.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.34.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.34.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.35.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.35.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.36.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.36.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.37.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.37.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.38.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.38.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.39.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.39.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.40.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.40.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.41.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.41.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.42.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.42.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.43.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.43.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.44.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.44.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.45.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.45.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.46.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.46.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.47.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.47.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.48.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.48.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.49.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.49.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.50.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.50.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.51.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.51.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.52.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.52.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.53.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.53.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.54.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.54.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.55.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.55.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.56.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.56.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.57.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.57.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.58.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.58.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.59.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.59.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.60.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.60.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.61.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.61.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.62.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.62.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.63.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.63.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.64.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.64.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.65.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.65.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.66.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.66.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.67.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.67.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.68.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.68.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.69.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.69.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.70.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.70.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.71.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.71.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.72.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.72.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.73.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.73.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.74.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.74.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.75.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.75.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.76.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.76.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.77.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.77.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.78.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.78.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.79.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.79.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.80.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.80.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.81.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.81.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.82.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.82.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.83.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.83.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.84.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.84.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.85.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.85.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.86.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.86.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.87.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.87.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.88.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.88.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.89.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.89.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.90.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.90.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.91.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.91.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.92.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.92.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.93.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.93.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.94.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.94.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.95.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.95.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.96.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.96.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.97.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.97.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.98.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.98.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.99.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.99.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.100.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.100.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.101.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.101.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.102.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.102.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.103.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.103.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.104.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.104.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.105.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.105.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.106.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.106.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.107.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.107.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.108.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.108.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.109.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.109.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.110.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.110.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.111.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.111.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.112.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.112.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.113.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.113.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.114.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.114.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.115.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.115.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.116.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.116.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.117.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.117.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.118.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.118.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.119.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.119.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.120.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.120.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.121.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.121.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.122.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.122.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.123.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.123.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.124.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.124.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.125.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.125.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.126.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.126.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.127.gate_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.127.up_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.0.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.1.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.2.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.3.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.4.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.5.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.6.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.7.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.8.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.9.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.10.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.11.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.12.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.13.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.14.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.15.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.16.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.17.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.18.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.19.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.20.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.21.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.22.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.23.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.24.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.25.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.26.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.27.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.28.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.29.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.30.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.31.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.32.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.33.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.34.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.35.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.36.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.37.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.38.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.39.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.40.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.41.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.42.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.43.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.44.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.45.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.46.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.47.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.48.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.49.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.50.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.51.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.52.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.53.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.54.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.55.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.56.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.57.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.58.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.59.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.60.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.61.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.62.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.63.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.64.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.65.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.66.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.67.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.68.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.69.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.70.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.71.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.72.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.73.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.74.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.75.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.76.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.77.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.78.down_proj.weight": "model-00029-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.79.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.80.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.81.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.82.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.83.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.84.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.85.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.86.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.87.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.88.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.89.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.90.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.91.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.92.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.93.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.94.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.95.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.96.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.97.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.98.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.99.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.100.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.101.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.102.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.103.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.104.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.105.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.106.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.107.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.108.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.109.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.110.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.111.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.112.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.113.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.114.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.115.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.116.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.117.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.118.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.119.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.120.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.121.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.122.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.123.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.124.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.125.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.126.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.experts.127.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.shared_experts.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.shared_experts.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.gate.e_score_correction_bias": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.shared_experts.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.mlp.gate.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.q_proj.bias": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.k_proj.bias": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.v_proj.bias": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.post_attention_layernorm.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.input_layernorm.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.q_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.k_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.v_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.33.self_attn.o_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.0.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.0.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.1.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.1.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.2.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.2.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.3.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.3.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.4.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.4.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.5.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.5.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.6.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.6.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.7.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.7.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.8.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.8.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.9.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.9.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.10.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.10.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.11.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.11.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.12.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.12.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.13.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.13.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.14.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.14.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.15.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.15.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.16.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.16.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.17.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.17.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.18.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.18.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.19.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.19.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.20.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.20.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.21.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.21.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.22.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.22.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.23.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.23.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.24.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.24.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.25.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.25.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.26.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.26.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.27.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.27.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.28.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.28.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.29.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.29.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.30.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.30.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.31.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.31.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.32.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.32.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.33.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.33.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.34.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.34.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.35.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.35.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.36.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.36.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.37.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.37.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.38.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.38.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.39.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.39.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.40.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.40.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.41.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.41.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.42.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.42.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.43.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.43.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.44.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.44.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.45.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.45.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.46.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.46.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.47.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.47.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.48.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.48.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.49.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.49.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.50.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.50.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.51.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.51.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.52.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.52.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.53.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.53.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.54.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.54.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.55.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.55.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.56.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.56.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.57.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.57.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.58.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.58.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.59.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.59.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.60.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.60.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.61.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.61.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.62.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.62.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.63.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.63.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.64.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.64.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.65.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.65.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.66.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.66.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.67.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.67.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.68.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.68.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.69.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.69.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.70.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.70.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.71.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.71.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.72.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.72.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.73.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.73.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.74.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.74.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.75.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.75.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.76.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.76.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.77.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.77.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.78.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.78.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.79.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.79.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.80.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.80.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.81.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.81.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.82.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.82.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.83.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.83.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.84.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.84.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.85.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.85.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.86.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.86.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.87.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.87.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.88.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.88.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.89.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.89.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.90.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.90.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.91.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.91.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.92.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.92.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.93.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.93.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.94.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.94.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.95.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.95.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.96.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.96.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.97.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.97.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.98.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.98.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.99.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.99.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.100.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.100.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.101.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.101.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.102.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.102.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.103.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.103.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.104.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.104.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.105.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.105.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.106.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.106.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.107.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.107.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.108.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.108.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.109.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.109.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.110.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.110.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.111.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.111.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.112.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.112.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.113.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.113.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.114.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.114.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.115.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.115.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.116.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.116.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.117.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.117.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.118.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.118.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.119.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.119.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.120.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.120.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.121.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.121.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.122.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.122.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.123.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.123.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.124.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.124.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.125.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.125.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.126.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.126.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.127.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.127.up_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.0.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.1.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.2.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.3.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.4.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.5.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.6.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.7.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.8.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.9.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.10.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.11.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.12.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.13.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.14.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.15.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.16.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.17.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.18.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.19.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.20.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.21.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.22.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.23.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.24.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.25.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.26.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.27.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.28.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.29.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.30.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.31.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.32.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.33.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.34.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.35.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.36.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.37.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.38.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.39.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.40.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.41.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.42.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.43.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.44.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.45.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.46.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.47.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.48.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.49.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.50.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.51.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.52.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.53.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.54.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.55.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.56.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.57.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.58.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.59.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.60.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.61.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.62.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.63.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.64.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.65.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.66.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.67.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.68.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.69.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.70.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.71.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.72.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.73.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.74.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.75.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.76.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.77.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.78.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.79.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.80.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.81.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.82.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.83.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.84.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.85.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.86.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.87.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.88.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.89.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.90.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.91.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.92.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.93.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.94.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.95.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.96.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.97.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.98.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.99.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.100.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.101.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.102.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.103.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.104.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.105.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.106.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.107.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.108.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.109.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.110.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.111.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.112.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.113.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.114.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.115.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.116.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.117.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.118.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.119.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.120.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.121.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.122.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.123.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.124.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.125.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.126.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.experts.127.down_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.o_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.shared_experts.gate_proj.weight": "model-00030-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.shared_experts.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.shared_experts.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.gate.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.mlp.gate.e_score_correction_bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.q_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.k_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.v_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.q_proj.bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.k_proj.bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.self_attn.v_proj.bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.post_attention_layernorm.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.34.input_layernorm.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.0.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.0.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.1.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.1.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.2.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.2.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.3.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.3.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.4.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.4.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.5.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.5.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.6.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.6.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.7.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.7.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.8.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.8.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.9.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.9.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.10.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.10.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.11.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.11.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.12.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.12.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.13.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.13.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.14.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.14.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.15.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.15.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.16.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.16.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.17.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.17.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.18.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.18.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.19.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.19.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.20.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.20.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.21.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.21.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.22.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.22.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.23.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.23.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.24.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.24.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.25.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.25.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.26.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.26.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.27.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.27.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.28.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.28.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.29.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.29.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.30.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.30.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.31.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.31.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.32.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.32.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.33.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.33.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.34.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.34.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.35.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.35.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.36.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.36.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.37.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.37.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.38.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.38.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.39.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.39.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.40.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.40.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.41.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.41.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.42.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.42.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.43.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.43.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.44.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.44.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.45.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.45.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.46.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.46.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.47.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.47.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.48.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.48.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.49.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.49.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.50.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.50.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.51.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.51.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.52.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.52.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.53.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.53.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.54.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.54.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.55.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.55.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.56.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.56.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.57.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.57.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.58.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.58.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.59.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.59.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.60.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.60.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.61.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.61.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.62.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.62.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.63.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.63.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.64.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.64.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.65.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.65.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.66.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.66.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.67.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.67.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.68.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.68.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.69.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.69.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.70.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.70.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.71.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.71.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.72.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.72.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.73.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.73.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.74.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.74.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.75.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.75.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.76.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.76.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.77.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.77.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.78.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.78.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.79.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.79.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.80.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.80.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.81.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.81.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.82.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.82.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.83.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.83.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.84.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.84.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.85.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.85.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.86.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.86.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.87.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.87.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.88.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.88.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.89.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.89.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.90.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.90.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.91.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.91.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.92.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.92.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.93.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.93.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.94.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.94.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.95.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.95.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.96.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.96.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.97.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.97.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.98.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.98.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.99.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.99.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.100.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.100.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.101.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.101.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.102.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.102.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.103.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.103.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.104.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.104.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.105.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.105.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.106.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.106.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.107.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.107.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.108.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.108.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.109.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.109.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.110.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.110.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.111.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.111.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.112.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.112.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.113.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.113.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.114.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.114.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.115.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.115.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.116.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.116.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.117.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.117.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.118.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.118.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.119.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.119.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.120.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.120.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.121.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.121.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.122.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.122.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.123.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.123.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.124.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.124.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.125.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.125.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.126.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.126.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.127.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.127.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.0.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.1.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.2.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.3.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.4.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.5.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.6.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.7.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.8.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.9.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.10.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.11.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.12.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.13.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.14.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.15.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.16.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.17.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.18.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.19.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.20.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.21.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.22.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.23.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.24.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.25.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.26.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.27.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.28.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.29.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.30.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.31.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.32.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.33.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.34.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.35.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.36.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.37.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.38.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.39.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.40.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.41.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.42.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.43.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.44.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.45.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.46.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.47.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.48.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.49.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.50.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.51.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.52.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.53.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.54.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.55.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.56.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.57.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.58.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.59.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.60.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.61.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.62.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.63.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.64.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.65.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.66.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.67.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.68.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.69.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.70.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.71.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.72.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.73.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.74.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.75.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.76.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.77.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.78.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.79.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.80.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.81.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.82.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.83.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.84.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.85.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.86.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.87.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.88.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.89.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.90.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.91.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.92.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.93.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.94.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.95.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.96.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.97.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.98.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.99.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.100.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.101.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.102.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.103.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.104.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.105.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.106.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.107.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.108.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.109.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.110.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.111.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.112.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.113.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.114.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.115.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.116.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.117.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.118.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.119.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.120.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.121.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.122.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.123.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.124.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.125.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.126.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.experts.127.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.o_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.post_attention_layernorm.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.shared_experts.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.shared_experts.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.shared_experts.down_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.gate.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.input_layernorm.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.mlp.gate.e_score_correction_bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.q_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.k_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.v_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.q_proj.bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.k_proj.bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.35.self_attn.v_proj.bias": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.0.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.0.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.1.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.1.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.2.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.2.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.3.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.3.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.4.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.4.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.5.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.5.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.6.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.6.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.7.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.7.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.8.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.8.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.9.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.9.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.10.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.10.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.11.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.11.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.12.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.12.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.13.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.13.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.14.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.14.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.15.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.15.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.16.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.16.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.17.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.17.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.18.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.18.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.19.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.19.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.20.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.20.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.21.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.21.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.22.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.22.up_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.23.gate_proj.weight": "model-00031-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.23.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.24.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.24.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.25.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.25.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.26.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.26.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.27.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.27.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.28.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.28.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.29.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.29.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.30.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.30.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.31.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.31.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.32.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.32.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.33.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.33.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.34.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.34.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.35.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.35.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.36.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.36.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.37.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.37.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.38.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.38.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.39.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.39.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.40.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.40.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.41.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.41.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.42.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.42.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.43.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.43.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.44.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.44.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.45.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.45.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.46.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.46.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.47.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.47.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.48.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.48.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.49.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.49.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.50.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.50.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.51.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.51.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.52.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.52.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.53.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.53.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.54.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.54.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.55.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.55.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.56.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.56.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.57.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.57.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.58.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.58.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.59.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.59.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.60.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.60.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.61.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.61.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.62.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.62.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.63.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.63.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.64.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.64.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.65.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.65.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.66.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.66.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.67.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.67.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.68.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.68.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.69.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.69.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.70.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.70.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.71.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.71.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.72.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.72.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.73.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.73.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.74.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.74.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.75.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.75.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.76.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.76.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.77.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.77.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.78.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.78.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.79.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.79.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.80.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.80.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.81.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.81.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.82.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.82.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.83.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.83.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.84.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.84.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.85.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.85.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.86.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.86.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.87.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.87.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.88.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.88.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.89.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.89.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.90.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.90.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.91.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.91.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.92.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.92.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.93.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.93.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.94.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.94.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.95.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.95.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.96.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.96.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.97.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.97.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.98.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.98.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.99.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.99.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.100.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.100.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.101.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.101.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.102.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.102.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.103.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.103.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.104.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.104.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.105.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.105.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.106.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.106.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.107.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.107.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.108.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.108.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.109.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.109.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.110.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.110.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.111.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.111.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.112.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.112.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.113.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.113.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.114.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.114.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.115.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.115.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.116.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.116.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.117.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.117.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.118.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.118.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.119.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.119.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.120.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.120.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.121.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.121.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.122.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.122.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.123.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.123.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.124.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.124.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.125.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.125.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.126.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.126.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.127.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.127.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.0.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.1.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.2.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.3.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.4.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.5.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.6.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.7.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.8.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.9.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.10.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.11.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.12.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.13.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.14.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.15.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.16.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.17.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.18.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.19.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.20.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.21.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.22.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.23.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.24.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.25.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.26.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.27.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.28.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.29.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.30.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.31.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.32.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.33.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.34.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.35.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.36.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.37.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.38.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.39.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.40.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.41.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.42.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.43.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.44.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.45.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.46.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.47.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.48.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.49.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.50.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.51.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.52.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.53.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.54.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.55.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.56.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.57.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.58.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.59.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.60.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.61.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.62.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.63.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.64.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.65.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.66.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.67.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.68.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.69.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.70.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.71.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.72.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.73.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.74.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.75.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.76.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.77.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.78.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.79.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.80.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.81.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.82.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.83.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.84.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.85.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.86.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.87.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.88.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.89.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.90.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.91.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.92.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.93.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.94.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.95.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.96.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.97.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.98.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.99.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.100.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.101.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.102.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.103.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.104.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.105.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.106.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.107.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.108.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.109.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.110.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.111.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.112.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.113.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.114.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.115.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.116.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.117.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.118.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.119.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.120.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.121.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.122.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.123.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.124.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.125.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.126.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.experts.127.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.q_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.k_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.v_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.o_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.shared_experts.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.shared_experts.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.gate.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.shared_experts.down_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.q_proj.bias": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.k_proj.bias": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.self_attn.v_proj.bias": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.post_attention_layernorm.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.input_layernorm.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.36.mlp.gate.e_score_correction_bias": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.0.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.0.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.1.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.1.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.2.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.2.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.3.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.3.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.4.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.4.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.5.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.5.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.6.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.6.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.7.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.7.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.8.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.8.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.9.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.9.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.10.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.10.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.11.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.11.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.12.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.12.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.13.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.13.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.14.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.14.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.15.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.15.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.16.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.16.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.17.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.17.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.18.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.18.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.19.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.19.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.20.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.20.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.21.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.21.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.22.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.22.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.23.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.23.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.24.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.24.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.25.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.25.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.26.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.26.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.27.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.27.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.28.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.28.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.29.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.29.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.30.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.30.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.31.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.31.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.32.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.32.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.33.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.33.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.34.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.34.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.35.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.35.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.36.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.36.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.37.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.37.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.38.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.38.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.39.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.39.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.40.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.40.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.41.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.41.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.42.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.42.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.43.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.43.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.44.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.44.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.45.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.45.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.46.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.46.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.47.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.47.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.48.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.48.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.49.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.49.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.50.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.50.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.51.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.51.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.52.gate_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.52.up_proj.weight": "model-00032-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.53.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.53.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.54.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.54.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.55.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.55.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.56.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.56.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.57.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.57.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.58.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.58.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.59.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.59.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.60.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.60.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.61.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.61.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.62.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.62.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.63.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.63.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.64.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.64.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.65.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.65.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.66.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.66.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.67.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.67.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.68.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.68.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.69.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.69.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.70.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.70.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.71.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.71.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.72.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.72.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.73.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.73.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.74.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.74.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.75.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.75.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.76.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.76.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.77.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.77.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.78.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.78.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.79.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.79.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.80.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.80.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.81.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.81.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.82.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.82.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.83.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.83.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.84.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.84.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.85.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.85.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.86.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.86.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.87.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.87.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.88.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.88.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.89.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.89.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.90.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.90.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.91.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.91.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.92.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.92.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.93.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.93.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.94.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.94.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.95.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.95.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.96.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.96.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.97.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.97.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.98.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.98.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.99.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.99.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.100.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.100.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.101.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.101.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.102.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.102.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.103.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.103.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.104.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.104.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.105.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.105.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.106.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.106.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.107.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.107.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.108.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.108.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.109.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.109.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.110.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.110.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.111.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.111.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.112.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.112.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.113.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.113.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.114.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.114.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.115.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.115.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.116.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.116.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.117.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.117.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.118.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.118.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.119.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.119.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.120.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.120.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.121.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.121.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.122.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.122.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.123.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.123.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.124.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.124.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.125.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.125.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.126.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.126.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.127.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.127.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.0.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.1.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.2.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.3.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.4.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.5.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.6.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.7.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.8.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.9.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.10.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.11.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.12.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.13.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.14.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.15.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.16.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.17.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.18.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.19.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.20.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.21.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.22.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.23.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.24.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.25.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.26.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.27.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.28.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.29.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.30.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.31.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.32.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.33.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.34.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.35.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.36.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.37.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.38.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.39.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.40.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.41.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.42.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.43.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.44.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.45.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.46.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.47.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.48.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.49.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.50.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.51.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.52.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.53.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.54.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.55.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.56.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.57.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.58.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.59.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.60.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.61.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.62.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.63.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.64.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.65.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.66.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.67.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.68.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.69.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.70.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.71.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.72.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.73.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.74.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.75.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.76.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.77.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.78.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.79.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.80.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.81.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.82.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.83.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.84.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.85.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.86.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.87.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.88.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.89.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.90.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.91.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.92.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.93.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.94.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.95.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.96.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.97.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.98.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.99.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.100.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.101.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.102.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.103.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.104.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.105.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.106.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.107.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.108.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.109.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.110.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.111.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.112.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.113.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.114.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.115.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.116.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.117.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.118.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.119.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.120.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.121.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.122.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.123.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.124.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.125.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.126.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.experts.127.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.shared_experts.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.shared_experts.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.q_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.k_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.v_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.shared_experts.down_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.o_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.input_layernorm.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.gate.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.q_proj.bias": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.k_proj.bias": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.self_attn.v_proj.bias": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.post_attention_layernorm.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.37.mlp.gate.e_score_correction_bias": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.0.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.0.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.1.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.1.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.2.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.2.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.3.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.3.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.4.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.4.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.5.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.5.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.6.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.6.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.7.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.7.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.8.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.8.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.9.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.9.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.10.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.10.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.11.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.11.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.12.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.12.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.13.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.13.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.14.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.14.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.15.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.15.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.16.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.16.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.17.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.17.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.18.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.18.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.19.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.19.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.20.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.20.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.21.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.21.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.22.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.22.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.23.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.23.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.24.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.24.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.25.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.25.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.26.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.26.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.27.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.27.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.28.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.28.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.29.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.29.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.30.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.30.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.31.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.31.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.32.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.32.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.33.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.33.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.34.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.34.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.35.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.35.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.36.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.36.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.37.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.37.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.38.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.38.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.39.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.39.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.40.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.40.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.41.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.41.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.42.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.42.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.43.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.43.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.44.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.44.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.45.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.45.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.46.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.46.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.47.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.47.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.48.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.48.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.49.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.49.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.50.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.50.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.51.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.51.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.52.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.52.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.53.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.53.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.54.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.54.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.55.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.55.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.56.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.56.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.57.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.57.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.58.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.58.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.59.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.59.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.60.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.60.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.61.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.61.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.62.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.62.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.63.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.63.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.64.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.64.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.65.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.65.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.66.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.66.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.67.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.67.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.68.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.68.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.69.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.69.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.70.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.70.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.71.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.71.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.72.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.72.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.73.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.73.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.74.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.74.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.75.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.75.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.76.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.76.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.77.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.77.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.78.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.78.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.79.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.79.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.80.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.80.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.81.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.81.up_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.82.gate_proj.weight": "model-00033-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.82.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.83.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.83.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.84.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.84.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.85.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.85.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.86.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.86.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.87.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.87.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.88.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.88.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.89.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.89.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.90.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.90.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.91.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.91.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.92.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.92.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.93.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.93.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.94.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.94.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.95.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.95.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.96.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.96.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.97.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.97.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.98.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.98.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.99.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.99.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.100.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.100.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.101.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.101.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.102.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.102.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.103.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.103.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.104.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.104.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.105.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.105.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.106.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.106.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.107.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.107.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.108.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.108.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.109.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.109.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.110.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.110.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.111.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.111.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.112.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.112.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.113.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.113.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.114.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.114.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.115.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.115.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.116.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.116.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.117.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.117.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.118.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.118.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.119.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.119.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.120.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.120.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.121.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.121.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.122.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.122.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.123.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.123.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.124.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.124.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.125.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.125.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.126.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.126.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.127.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.127.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.0.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.1.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.2.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.3.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.4.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.5.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.6.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.7.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.8.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.9.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.10.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.11.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.12.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.13.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.14.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.15.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.16.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.17.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.18.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.19.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.20.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.21.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.22.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.23.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.24.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.25.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.26.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.27.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.28.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.29.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.30.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.31.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.32.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.33.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.34.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.35.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.36.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.37.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.38.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.39.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.40.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.41.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.42.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.43.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.44.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.45.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.46.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.47.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.48.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.49.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.50.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.51.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.52.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.53.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.54.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.55.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.56.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.57.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.58.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.59.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.60.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.61.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.62.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.63.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.64.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.65.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.66.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.67.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.68.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.69.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.70.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.71.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.72.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.73.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.74.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.75.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.76.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.77.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.78.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.79.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.80.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.81.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.82.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.83.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.84.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.85.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.86.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.87.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.88.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.89.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.90.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.91.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.92.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.93.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.94.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.95.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.96.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.97.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.98.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.99.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.100.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.101.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.102.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.103.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.104.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.105.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.106.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.107.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.108.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.109.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.110.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.111.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.112.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.113.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.114.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.115.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.116.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.117.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.118.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.119.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.120.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.121.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.122.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.123.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.124.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.125.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.126.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.experts.127.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.q_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.k_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.v_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.o_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.gate.e_score_correction_bias": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.shared_experts.down_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.post_attention_layernorm.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.gate.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.shared_experts.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.mlp.shared_experts.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.input_layernorm.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.q_proj.bias": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.k_proj.bias": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.38.self_attn.v_proj.bias": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.0.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.0.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.1.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.1.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.2.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.2.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.3.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.3.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.4.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.4.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.5.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.5.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.6.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.6.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.7.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.7.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.8.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.8.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.9.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.9.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.10.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.10.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.11.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.11.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.12.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.12.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.13.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.13.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.14.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.14.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.15.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.15.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.16.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.16.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.17.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.17.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.18.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.18.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.19.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.19.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.20.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.20.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.21.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.21.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.22.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.22.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.23.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.23.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.24.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.24.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.25.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.25.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.26.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.26.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.27.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.27.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.28.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.28.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.29.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.29.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.30.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.30.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.31.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.31.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.32.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.32.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.33.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.33.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.34.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.34.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.35.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.35.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.36.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.36.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.37.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.37.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.38.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.38.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.39.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.39.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.40.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.40.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.41.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.41.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.42.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.42.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.43.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.43.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.44.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.44.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.45.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.45.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.46.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.46.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.47.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.47.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.48.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.48.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.49.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.49.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.50.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.50.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.51.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.51.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.52.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.52.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.53.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.53.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.54.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.54.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.55.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.55.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.56.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.56.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.57.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.57.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.58.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.58.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.59.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.59.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.60.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.60.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.61.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.61.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.62.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.62.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.63.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.63.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.64.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.64.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.65.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.65.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.66.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.66.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.67.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.67.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.68.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.68.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.69.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.69.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.70.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.70.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.71.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.71.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.72.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.72.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.73.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.73.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.74.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.74.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.75.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.75.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.76.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.76.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.77.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.77.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.78.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.78.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.79.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.79.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.80.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.80.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.81.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.81.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.82.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.82.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.83.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.83.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.84.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.84.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.85.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.85.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.86.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.86.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.87.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.87.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.88.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.88.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.89.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.89.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.90.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.90.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.91.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.91.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.92.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.92.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.93.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.93.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.94.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.94.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.95.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.95.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.96.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.96.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.97.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.97.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.98.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.98.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.99.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.99.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.100.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.100.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.101.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.101.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.102.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.102.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.103.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.103.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.104.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.104.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.105.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.105.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.106.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.106.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.107.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.107.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.108.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.108.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.109.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.109.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.110.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.110.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.111.gate_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.111.up_proj.weight": "model-00034-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.112.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.112.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.113.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.113.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.114.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.114.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.115.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.115.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.116.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.116.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.117.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.117.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.118.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.118.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.119.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.119.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.120.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.120.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.121.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.121.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.122.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.122.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.123.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.123.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.124.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.124.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.125.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.125.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.126.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.126.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.127.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.127.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.0.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.1.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.2.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.3.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.4.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.5.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.6.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.7.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.8.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.9.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.10.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.11.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.12.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.13.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.14.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.15.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.16.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.17.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.18.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.19.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.20.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.21.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.22.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.23.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.24.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.25.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.26.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.27.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.28.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.29.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.30.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.31.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.32.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.33.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.34.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.35.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.36.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.37.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.38.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.39.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.40.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.41.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.42.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.43.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.44.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.45.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.46.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.47.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.48.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.49.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.50.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.51.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.52.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.53.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.54.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.55.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.56.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.57.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.58.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.59.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.60.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.61.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.62.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.63.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.64.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.65.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.66.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.67.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.68.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.69.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.70.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.71.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.72.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.73.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.74.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.75.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.76.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.77.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.78.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.79.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.80.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.81.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.82.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.83.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.84.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.85.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.86.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.87.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.88.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.89.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.90.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.91.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.92.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.93.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.94.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.95.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.96.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.97.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.98.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.99.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.100.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.101.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.102.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.103.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.104.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.105.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.106.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.107.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.108.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.109.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.110.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.111.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.112.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.113.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.114.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.115.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.116.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.117.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.118.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.119.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.120.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.121.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.122.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.123.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.124.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.125.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.126.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.experts.127.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.q_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.k_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.v_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.o_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.shared_experts.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.shared_experts.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.shared_experts.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.q_proj.bias": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.k_proj.bias": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.self_attn.v_proj.bias": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.gate.e_score_correction_bias": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.mlp.gate.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.post_attention_layernorm.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.39.input_layernorm.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.0.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.0.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.1.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.1.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.2.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.2.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.3.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.3.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.4.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.4.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.5.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.5.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.6.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.6.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.7.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.7.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.8.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.8.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.9.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.9.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.10.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.10.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.11.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.11.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.12.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.12.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.13.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.13.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.14.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.14.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.15.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.15.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.16.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.16.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.17.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.17.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.18.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.18.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.19.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.19.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.20.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.20.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.21.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.21.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.22.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.22.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.23.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.23.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.24.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.24.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.25.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.25.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.26.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.26.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.27.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.27.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.28.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.28.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.29.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.29.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.30.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.30.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.31.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.31.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.32.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.32.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.33.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.33.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.34.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.34.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.35.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.35.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.36.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.36.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.37.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.37.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.38.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.38.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.39.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.39.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.40.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.40.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.41.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.41.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.42.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.42.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.43.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.43.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.44.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.44.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.45.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.45.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.46.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.46.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.47.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.47.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.48.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.48.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.49.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.49.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.50.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.50.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.51.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.51.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.52.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.52.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.53.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.53.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.54.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.54.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.55.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.55.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.56.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.56.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.57.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.57.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.58.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.58.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.59.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.59.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.60.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.60.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.61.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.61.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.62.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.62.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.63.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.63.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.64.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.64.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.65.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.65.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.66.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.66.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.67.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.67.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.68.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.68.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.69.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.69.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.70.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.70.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.71.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.71.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.72.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.72.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.73.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.73.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.74.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.74.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.75.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.75.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.76.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.76.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.77.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.77.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.78.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.78.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.79.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.79.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.80.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.80.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.81.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.81.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.82.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.82.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.83.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.83.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.84.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.84.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.85.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.85.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.86.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.86.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.87.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.87.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.88.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.88.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.89.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.89.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.90.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.90.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.91.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.91.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.92.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.92.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.93.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.93.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.94.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.94.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.95.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.95.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.96.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.96.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.97.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.97.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.98.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.98.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.99.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.99.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.100.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.100.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.101.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.101.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.102.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.102.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.103.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.103.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.104.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.104.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.105.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.105.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.106.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.106.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.107.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.107.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.108.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.108.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.109.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.109.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.110.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.110.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.111.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.111.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.112.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.112.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.113.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.113.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.114.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.114.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.115.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.115.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.116.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.116.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.117.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.117.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.118.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.118.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.119.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.119.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.120.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.120.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.121.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.121.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.122.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.122.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.123.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.123.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.124.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.124.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.125.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.125.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.126.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.126.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.127.gate_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.127.up_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.0.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.1.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.2.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.3.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.4.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.5.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.6.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.7.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.8.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.9.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.10.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.11.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.12.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.13.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.14.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.15.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.16.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.17.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.18.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.19.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.20.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.21.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.22.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.23.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.24.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.25.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.26.down_proj.weight": "model-00035-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.27.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.28.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.29.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.30.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.31.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.32.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.33.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.34.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.35.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.36.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.37.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.38.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.39.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.40.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.41.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.42.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.43.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.44.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.45.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.46.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.47.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.48.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.49.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.50.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.51.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.52.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.53.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.54.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.55.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.56.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.57.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.58.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.59.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.60.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.61.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.62.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.63.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.64.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.65.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.66.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.67.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.68.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.69.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.70.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.71.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.72.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.73.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.74.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.75.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.76.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.77.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.78.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.79.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.80.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.81.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.82.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.83.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.84.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.85.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.86.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.87.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.88.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.89.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.90.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.91.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.92.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.93.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.94.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.95.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.96.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.97.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.98.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.99.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.100.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.101.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.102.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.103.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.104.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.105.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.106.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.107.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.108.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.109.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.110.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.111.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.112.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.113.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.114.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.115.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.116.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.117.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.118.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.119.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.120.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.121.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.122.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.123.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.124.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.125.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.126.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.experts.127.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.gate.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.q_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.k_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.v_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.o_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.shared_experts.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.shared_experts.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.post_attention_layernorm.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.shared_experts.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.input_layernorm.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.mlp.gate.e_score_correction_bias": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.q_proj.bias": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.k_proj.bias": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.40.self_attn.v_proj.bias": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.0.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.0.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.1.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.1.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.2.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.2.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.3.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.3.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.4.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.4.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.5.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.5.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.6.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.6.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.7.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.7.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.8.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.8.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.9.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.9.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.10.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.10.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.11.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.11.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.12.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.12.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.13.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.13.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.14.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.14.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.15.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.15.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.16.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.16.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.17.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.17.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.18.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.18.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.19.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.19.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.20.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.20.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.21.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.21.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.22.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.22.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.23.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.23.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.24.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.24.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.25.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.25.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.26.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.26.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.27.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.27.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.28.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.28.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.29.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.29.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.30.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.30.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.31.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.31.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.32.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.32.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.33.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.33.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.34.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.34.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.35.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.35.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.36.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.36.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.37.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.37.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.38.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.38.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.39.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.39.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.40.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.40.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.41.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.41.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.42.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.42.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.43.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.43.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.44.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.44.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.45.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.45.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.46.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.46.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.47.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.47.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.48.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.48.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.49.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.49.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.50.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.50.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.51.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.51.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.52.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.52.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.53.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.53.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.54.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.54.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.55.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.55.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.56.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.56.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.57.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.57.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.58.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.58.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.59.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.59.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.60.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.60.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.61.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.61.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.62.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.62.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.63.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.63.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.64.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.64.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.65.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.65.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.66.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.66.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.67.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.67.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.68.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.68.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.69.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.69.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.70.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.70.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.71.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.71.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.72.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.72.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.73.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.73.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.74.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.74.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.75.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.75.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.76.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.76.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.77.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.77.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.78.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.78.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.79.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.79.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.80.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.80.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.81.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.81.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.82.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.82.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.83.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.83.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.84.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.84.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.85.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.85.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.86.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.86.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.87.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.87.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.88.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.88.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.89.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.89.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.90.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.90.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.91.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.91.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.92.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.92.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.93.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.93.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.94.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.94.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.95.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.95.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.96.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.96.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.97.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.97.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.98.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.98.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.99.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.99.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.100.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.100.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.101.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.101.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.102.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.102.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.103.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.103.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.104.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.104.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.105.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.105.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.106.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.106.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.107.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.107.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.108.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.108.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.109.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.109.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.110.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.110.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.111.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.111.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.112.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.112.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.113.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.113.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.114.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.114.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.115.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.115.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.116.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.116.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.117.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.117.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.118.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.118.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.119.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.119.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.120.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.120.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.121.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.121.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.122.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.122.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.123.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.123.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.124.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.124.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.125.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.125.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.126.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.126.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.127.gate_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.127.up_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.0.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.1.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.2.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.3.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.4.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.5.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.6.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.7.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.8.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.9.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.10.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.11.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.12.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.13.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.14.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.15.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.16.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.17.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.18.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.19.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.20.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.21.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.22.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.23.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.24.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.25.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.26.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.27.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.28.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.29.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.30.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.31.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.32.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.33.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.34.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.35.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.36.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.37.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.38.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.39.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.40.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.41.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.42.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.43.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.44.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.45.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.46.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.47.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.48.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.49.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.50.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.51.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.52.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.53.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.54.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.55.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.56.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.57.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.58.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.59.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.60.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.61.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.62.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.63.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.64.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.65.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.66.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.67.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.68.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.69.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.70.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.71.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.72.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.73.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.74.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.75.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.76.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.77.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.78.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.79.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.80.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.81.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.82.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.83.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.84.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.85.down_proj.weight": "model-00036-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.86.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.87.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.88.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.89.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.90.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.91.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.92.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.93.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.94.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.95.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.96.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.97.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.98.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.99.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.100.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.101.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.102.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.103.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.104.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.105.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.106.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.107.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.108.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.109.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.110.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.111.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.112.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.113.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.114.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.115.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.116.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.117.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.118.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.119.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.120.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.121.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.122.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.123.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.124.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.125.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.126.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.experts.127.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.gate.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.shared_experts.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.input_layernorm.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.q_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.k_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.v_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.o_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.q_proj.bias": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.k_proj.bias": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.self_attn.v_proj.bias": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.shared_experts.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.shared_experts.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.post_attention_layernorm.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.41.mlp.gate.e_score_correction_bias": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.0.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.0.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.1.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.1.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.2.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.2.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.3.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.3.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.4.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.4.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.5.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.5.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.6.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.6.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.7.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.7.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.8.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.8.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.9.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.9.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.10.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.10.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.11.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.11.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.12.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.12.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.13.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.13.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.14.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.14.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.15.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.15.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.16.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.16.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.17.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.17.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.18.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.18.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.19.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.19.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.20.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.20.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.21.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.21.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.22.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.22.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.23.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.23.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.24.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.24.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.25.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.25.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.26.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.26.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.27.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.27.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.28.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.28.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.29.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.29.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.30.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.30.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.31.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.31.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.32.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.32.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.33.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.33.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.34.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.34.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.35.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.35.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.36.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.36.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.37.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.37.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.38.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.38.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.39.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.39.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.40.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.40.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.41.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.41.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.42.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.42.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.43.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.43.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.44.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.44.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.45.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.45.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.46.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.46.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.47.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.47.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.48.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.48.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.49.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.49.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.50.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.50.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.51.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.51.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.52.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.52.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.53.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.53.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.54.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.54.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.55.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.55.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.56.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.56.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.57.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.57.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.58.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.58.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.59.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.59.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.60.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.60.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.61.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.61.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.62.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.62.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.63.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.63.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.64.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.64.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.65.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.65.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.66.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.66.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.67.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.67.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.68.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.68.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.69.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.69.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.70.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.70.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.71.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.71.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.72.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.72.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.73.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.73.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.74.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.74.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.75.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.75.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.76.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.76.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.77.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.77.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.78.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.78.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.79.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.79.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.80.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.80.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.81.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.81.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.82.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.82.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.83.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.83.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.84.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.84.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.85.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.85.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.86.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.86.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.87.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.87.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.88.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.88.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.89.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.89.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.90.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.90.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.91.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.91.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.92.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.92.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.93.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.93.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.94.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.94.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.95.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.95.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.96.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.96.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.97.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.97.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.98.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.98.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.99.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.99.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.100.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.100.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.101.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.101.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.102.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.102.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.103.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.103.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.104.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.104.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.105.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.105.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.106.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.106.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.107.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.107.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.108.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.108.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.109.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.109.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.110.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.110.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.111.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.111.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.112.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.112.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.113.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.113.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.114.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.114.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.115.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.115.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.116.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.116.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.117.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.117.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.118.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.118.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.119.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.119.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.120.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.120.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.121.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.121.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.122.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.122.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.123.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.123.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.124.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.124.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.125.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.125.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.126.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.126.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.127.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.127.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.0.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.1.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.2.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.3.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.4.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.5.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.6.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.7.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.8.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.9.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.10.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.11.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.12.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.13.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.14.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.15.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.16.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.17.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.18.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.19.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.20.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.21.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.22.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.23.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.24.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.25.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.26.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.27.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.28.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.29.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.30.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.31.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.32.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.33.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.34.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.35.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.36.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.37.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.38.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.39.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.40.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.41.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.42.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.43.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.44.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.45.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.46.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.47.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.48.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.49.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.50.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.51.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.52.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.53.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.54.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.55.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.56.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.57.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.58.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.59.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.60.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.61.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.62.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.63.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.64.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.65.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.66.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.67.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.68.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.69.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.70.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.71.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.72.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.73.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.74.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.75.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.76.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.77.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.78.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.79.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.80.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.81.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.82.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.83.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.84.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.85.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.86.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.87.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.88.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.89.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.90.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.91.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.92.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.93.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.94.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.95.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.96.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.97.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.98.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.99.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.100.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.101.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.102.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.103.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.104.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.105.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.106.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.107.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.108.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.109.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.110.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.111.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.112.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.113.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.114.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.115.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.116.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.117.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.118.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.119.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.120.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.121.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.122.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.123.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.124.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.125.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.126.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.experts.127.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.gate.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.shared_experts.gate_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.shared_experts.up_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.gate.e_score_correction_bias": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.mlp.shared_experts.down_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.q_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.k_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.v_proj.weight": "model-00037-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.o_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.42.post_attention_layernorm.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.42.input_layernorm.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.q_proj.bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.k_proj.bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.42.self_attn.v_proj.bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.0.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.0.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.1.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.1.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.2.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.2.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.3.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.3.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.4.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.4.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.5.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.5.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.6.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.6.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.7.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.7.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.8.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.8.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.9.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.9.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.10.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.10.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.11.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.11.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.12.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.12.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.13.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.13.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.14.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.14.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.15.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.15.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.16.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.16.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.17.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.17.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.18.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.18.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.19.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.19.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.20.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.20.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.21.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.21.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.22.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.22.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.23.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.23.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.24.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.24.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.25.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.25.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.26.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.26.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.27.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.27.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.28.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.28.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.29.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.29.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.30.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.30.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.31.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.31.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.32.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.32.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.33.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.33.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.34.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.34.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.35.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.35.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.36.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.36.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.37.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.37.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.38.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.38.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.39.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.39.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.40.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.40.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.41.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.41.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.42.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.42.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.43.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.43.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.44.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.44.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.45.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.45.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.46.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.46.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.47.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.47.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.48.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.48.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.49.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.49.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.50.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.50.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.51.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.51.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.52.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.52.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.53.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.53.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.54.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.54.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.55.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.55.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.56.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.56.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.57.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.57.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.58.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.58.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.59.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.59.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.60.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.60.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.61.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.61.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.62.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.62.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.63.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.63.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.64.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.64.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.65.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.65.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.66.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.66.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.67.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.67.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.68.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.68.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.69.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.69.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.70.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.70.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.71.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.71.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.72.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.72.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.73.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.73.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.74.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.74.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.75.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.75.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.76.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.76.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.77.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.77.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.78.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.78.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.79.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.79.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.80.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.80.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.81.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.81.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.82.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.82.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.83.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.83.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.84.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.84.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.85.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.85.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.86.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.86.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.87.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.87.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.88.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.88.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.89.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.89.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.90.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.90.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.91.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.91.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.92.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.92.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.93.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.93.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.94.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.94.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.95.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.95.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.96.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.96.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.97.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.97.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.98.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.98.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.99.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.99.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.100.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.100.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.101.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.101.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.102.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.102.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.103.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.103.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.104.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.104.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.105.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.105.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.106.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.106.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.107.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.107.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.108.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.108.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.109.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.109.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.110.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.110.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.111.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.111.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.112.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.112.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.113.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.113.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.114.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.114.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.115.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.115.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.116.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.116.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.117.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.117.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.118.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.118.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.119.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.119.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.120.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.120.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.121.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.121.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.122.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.122.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.123.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.123.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.124.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.124.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.125.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.125.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.126.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.126.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.127.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.127.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.0.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.1.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.2.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.3.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.4.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.5.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.6.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.7.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.8.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.9.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.10.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.11.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.12.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.13.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.14.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.15.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.16.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.17.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.18.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.19.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.20.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.21.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.22.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.23.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.24.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.25.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.26.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.27.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.28.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.29.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.30.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.31.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.32.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.33.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.34.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.35.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.36.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.37.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.38.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.39.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.40.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.41.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.42.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.43.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.44.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.45.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.46.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.47.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.48.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.49.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.50.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.51.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.52.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.53.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.54.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.55.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.56.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.57.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.58.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.59.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.60.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.61.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.62.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.63.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.64.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.65.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.66.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.67.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.68.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.69.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.70.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.71.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.72.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.73.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.74.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.75.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.76.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.77.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.78.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.79.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.80.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.81.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.82.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.83.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.84.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.85.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.86.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.87.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.88.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.89.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.90.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.91.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.92.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.93.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.94.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.95.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.96.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.97.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.98.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.99.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.100.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.101.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.102.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.103.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.104.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.105.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.106.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.107.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.108.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.109.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.110.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.111.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.112.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.113.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.114.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.115.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.116.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.117.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.118.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.119.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.120.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.121.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.122.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.123.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.124.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.125.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.126.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.experts.127.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.shared_experts.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.shared_experts.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.q_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.k_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.v_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.gate.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.shared_experts.down_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.o_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.mlp.gate.e_score_correction_bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.post_attention_layernorm.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.input_layernorm.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.q_proj.bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.k_proj.bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.43.self_attn.v_proj.bias": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.0.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.0.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.1.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.1.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.2.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.2.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.3.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.3.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.4.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.4.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.5.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.5.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.6.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.6.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.7.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.7.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.8.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.8.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.9.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.9.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.10.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.10.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.11.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.11.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.12.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.12.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.13.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.13.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.14.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.14.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.15.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.15.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.16.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.16.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.17.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.17.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.18.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.18.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.19.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.19.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.20.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.20.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.21.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.21.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.22.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.22.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.23.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.23.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.24.gate_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.24.up_proj.weight": "model-00038-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.25.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.25.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.26.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.26.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.27.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.27.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.28.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.28.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.29.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.29.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.30.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.30.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.31.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.31.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.32.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.32.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.33.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.33.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.34.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.34.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.35.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.35.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.36.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.36.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.37.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.37.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.38.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.38.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.39.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.39.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.40.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.40.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.41.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.41.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.42.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.42.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.43.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.43.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.44.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.44.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.45.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.45.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.46.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.46.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.47.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.47.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.48.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.48.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.49.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.49.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.50.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.50.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.51.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.51.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.52.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.52.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.53.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.53.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.54.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.54.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.55.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.55.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.56.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.56.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.57.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.57.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.58.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.58.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.59.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.59.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.60.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.60.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.61.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.61.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.62.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.62.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.63.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.63.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.64.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.64.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.65.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.65.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.66.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.66.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.67.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.67.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.68.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.68.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.69.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.69.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.70.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.70.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.71.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.71.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.72.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.72.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.73.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.73.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.74.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.74.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.75.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.75.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.76.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.76.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.77.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.77.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.78.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.78.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.79.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.79.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.80.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.80.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.81.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.81.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.82.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.82.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.83.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.83.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.84.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.84.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.85.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.85.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.86.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.86.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.87.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.87.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.88.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.88.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.89.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.89.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.90.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.90.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.91.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.91.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.92.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.92.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.93.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.93.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.94.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.94.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.95.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.95.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.96.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.96.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.97.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.97.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.98.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.98.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.99.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.99.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.100.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.100.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.101.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.101.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.102.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.102.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.103.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.103.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.104.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.104.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.105.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.105.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.106.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.106.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.107.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.107.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.108.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.108.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.109.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.109.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.110.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.110.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.111.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.111.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.112.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.112.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.113.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.113.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.114.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.114.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.115.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.115.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.116.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.116.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.117.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.117.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.118.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.118.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.119.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.119.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.120.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.120.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.121.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.121.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.122.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.122.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.123.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.123.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.124.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.124.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.125.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.125.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.126.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.126.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.127.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.127.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.0.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.1.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.2.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.3.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.4.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.5.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.6.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.7.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.8.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.9.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.10.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.11.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.12.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.13.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.14.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.15.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.16.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.17.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.18.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.19.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.20.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.21.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.22.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.23.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.24.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.25.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.26.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.27.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.28.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.29.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.30.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.31.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.32.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.33.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.34.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.35.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.36.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.37.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.38.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.39.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.40.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.41.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.42.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.43.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.44.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.45.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.46.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.47.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.48.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.49.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.50.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.51.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.52.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.53.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.54.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.55.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.56.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.57.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.58.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.59.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.60.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.61.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.62.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.63.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.64.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.65.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.66.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.67.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.68.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.69.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.70.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.71.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.72.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.73.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.74.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.75.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.76.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.77.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.78.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.79.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.80.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.81.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.82.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.83.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.84.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.85.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.86.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.87.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.88.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.89.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.90.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.91.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.92.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.93.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.94.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.95.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.96.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.97.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.98.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.99.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.100.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.101.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.102.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.103.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.104.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.105.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.106.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.107.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.108.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.109.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.110.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.111.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.112.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.113.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.114.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.115.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.116.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.117.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.118.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.119.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.120.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.121.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.122.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.123.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.124.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.125.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.126.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.experts.127.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.post_attention_layernorm.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.q_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.k_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.v_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.gate.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.shared_experts.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.shared_experts.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.o_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.shared_experts.down_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.input_layernorm.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.q_proj.bias": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.k_proj.bias": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.self_attn.v_proj.bias": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.44.mlp.gate.e_score_correction_bias": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.0.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.0.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.1.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.1.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.2.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.2.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.3.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.3.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.4.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.4.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.5.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.5.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.6.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.6.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.7.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.7.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.8.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.8.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.9.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.9.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.10.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.10.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.11.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.11.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.12.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.12.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.13.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.13.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.14.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.14.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.15.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.15.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.16.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.16.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.17.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.17.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.18.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.18.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.19.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.19.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.20.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.20.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.21.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.21.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.22.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.22.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.23.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.23.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.24.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.24.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.25.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.25.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.26.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.26.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.27.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.27.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.28.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.28.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.29.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.29.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.30.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.30.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.31.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.31.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.32.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.32.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.33.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.33.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.34.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.34.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.35.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.35.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.36.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.36.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.37.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.37.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.38.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.38.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.39.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.39.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.40.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.40.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.41.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.41.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.42.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.42.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.43.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.43.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.44.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.44.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.45.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.45.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.46.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.46.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.47.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.47.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.48.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.48.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.49.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.49.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.50.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.50.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.51.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.51.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.52.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.52.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.53.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.53.up_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.54.gate_proj.weight": "model-00039-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.54.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.55.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.55.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.56.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.56.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.57.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.57.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.58.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.58.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.59.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.59.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.60.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.60.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.61.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.61.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.62.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.62.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.63.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.63.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.64.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.64.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.65.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.65.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.66.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.66.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.67.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.67.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.68.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.68.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.69.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.69.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.70.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.70.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.71.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.71.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.72.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.72.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.73.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.73.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.74.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.74.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.75.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.75.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.76.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.76.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.77.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.77.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.78.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.78.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.79.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.79.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.80.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.80.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.81.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.81.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.82.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.82.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.83.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.83.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.84.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.84.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.85.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.85.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.86.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.86.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.87.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.87.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.88.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.88.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.89.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.89.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.90.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.90.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.91.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.91.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.92.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.92.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.93.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.93.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.94.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.94.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.95.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.95.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.96.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.96.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.97.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.97.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.98.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.98.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.99.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.99.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.100.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.100.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.101.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.101.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.102.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.102.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.103.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.103.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.104.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.104.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.105.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.105.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.106.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.106.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.107.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.107.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.108.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.108.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.109.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.109.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.110.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.110.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.111.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.111.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.112.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.112.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.113.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.113.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.114.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.114.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.115.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.115.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.116.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.116.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.117.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.117.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.118.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.118.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.119.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.119.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.120.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.120.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.121.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.121.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.122.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.122.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.123.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.123.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.124.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.124.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.125.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.125.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.126.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.126.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.127.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.127.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.0.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.1.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.2.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.3.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.4.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.5.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.6.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.7.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.8.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.9.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.10.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.11.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.12.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.13.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.14.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.15.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.16.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.17.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.18.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.19.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.20.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.21.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.22.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.23.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.24.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.25.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.26.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.27.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.28.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.29.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.30.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.31.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.32.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.33.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.34.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.35.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.36.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.37.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.38.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.39.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.40.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.41.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.42.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.43.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.44.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.45.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.46.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.47.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.48.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.49.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.50.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.51.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.52.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.53.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.54.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.55.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.56.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.57.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.58.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.59.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.60.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.61.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.62.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.63.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.64.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.65.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.66.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.67.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.68.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.69.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.70.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.71.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.72.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.73.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.74.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.75.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.76.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.77.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.78.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.79.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.80.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.81.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.82.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.83.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.84.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.85.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.86.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.87.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.88.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.89.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.90.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.91.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.92.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.93.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.94.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.95.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.96.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.97.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.98.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.99.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.100.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.101.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.102.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.103.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.104.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.105.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.106.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.107.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.108.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.109.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.110.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.111.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.112.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.113.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.114.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.115.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.116.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.117.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.118.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.119.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.120.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.121.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.122.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.123.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.124.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.125.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.126.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.experts.127.down_proj.weight": "model-00040-of-00041.safetensors",
+    "lm_head.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.q_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.k_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.v_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.gate.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.o_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.shared_experts.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.shared_experts.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.shared_experts.down_proj.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.post_attention_layernorm.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.input_layernorm.weight": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.q_proj.bias": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.k_proj.bias": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.self_attn.v_proj.bias": "model-00040-of-00041.safetensors",
+    "model.language_model.layers.45.mlp.gate.e_score_correction_bias": "model-00040-of-00041.safetensors",
+    "model.language_model.norm.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.0.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.0.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.1.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.1.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.2.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.2.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.3.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.3.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.4.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.4.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.5.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.5.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.6.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.6.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.7.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.7.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.8.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.8.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.9.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.9.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.10.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.10.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.11.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.11.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.12.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.12.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.13.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.13.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.14.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.14.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.15.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.15.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.16.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.16.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.17.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.17.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.18.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.18.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.19.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.19.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.20.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.20.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.21.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.21.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.22.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.22.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.23.mlp.gate_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.23.mlp.up_proj.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.0.attn.qkv.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.1.attn.qkv.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.2.attn.qkv.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.3.attn.qkv.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.4.attn.qkv.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.5.attn.qkv.weight": "model-00040-of-00041.safetensors",
+    "model.visual.blocks.6.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.7.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.8.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.9.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.10.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.11.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.12.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.13.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.14.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.15.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.16.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.17.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.18.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.19.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.20.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.21.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.22.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.23.attn.qkv.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.0.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.1.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.2.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.3.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.4.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.5.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.6.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.7.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.8.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.9.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.10.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.11.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.12.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.13.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.14.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.15.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.16.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.17.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.18.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.19.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.20.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.21.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.22.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.23.attn.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.0.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.1.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.2.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.3.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.4.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.5.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.6.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.7.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.8.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.9.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.10.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.11.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.12.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.13.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.14.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.15.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.16.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.17.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.18.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.19.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.20.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.21.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.22.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.23.mlp.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.patch_embed.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.0.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.1.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.2.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.3.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.4.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.5.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.6.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.7.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.8.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.9.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.10.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.11.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.12.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.13.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.14.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.15.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.16.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.17.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.18.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.19.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.20.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.21.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.22.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.23.norm1.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.0.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.1.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.2.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.3.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.4.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.5.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.6.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.7.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.8.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.9.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.10.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.11.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.12.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.13.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.14.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.15.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.16.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.17.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.18.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.19.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.20.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.21.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.22.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.blocks.23.norm2.weight": "model-00041-of-00041.safetensors",
+    "model.visual.merger.post_projection_norm.bias": "model-00041-of-00041.safetensors",
+    "model.visual.embeddings.position_embedding.weight": "model-00041-of-00041.safetensors",
+    "model.visual.merger.gate_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.merger.up_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.merger.down_proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.downsample.weight": "model-00041-of-00041.safetensors",
+    "model.visual.merger.post_projection_norm.weight": "model-00041-of-00041.safetensors",
+    "model.visual.post_conv_layernorm.weight": "model-00041-of-00041.safetensors",
+    "model.visual.patch_embed.proj.bias": "model-00041-of-00041.safetensors",
+    "model.visual.merger.proj.weight": "model-00041-of-00041.safetensors",
+    "model.visual.post_layernorm.weight": "model-00041-of-00041.safetensors",
+    "model.visual.downsample.bias": "model-00041-of-00041.safetensors"
+  }
+}
\ No newline at end of file
diff --git a/preprocessor_config.json b/preprocessor_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..308553695af766b3e3d05e68279d2c690e73273e
--- /dev/null
+++ b/preprocessor_config.json
@@ -0,0 +1,11 @@
+{
+    "size": {"shortest_edge": 12544, "longest_edge": 9633792},
+    "do_rescale": true,
+    "patch_size": 14,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [0.48145466, 0.4578275, 0.40821073],
+    "image_std": [0.26862954, 0.26130258, 0.27577711],
+    "image_processor_type": "Glm46VImageProcessor",
+    "processor_class": "Glm46VProcessor"
+}
diff --git a/special_tokens_map.json b/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..ff4ebdcfb4c2810ba6dd0ab115f5c0f9b83c92b1
--- /dev/null
+++ b/special_tokens_map.json
@@ -0,0 +1,42 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|image|>",
+    "<|video|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>",
+    "<|code_prefix|>",
+    "<|code_middle|>",
+    "<|code_suffix|>",
+    "/nothink"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/tokenizer.json b/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..902315d88b393628f0ab08d30b4a35d151fb631e
--- /dev/null
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2ff52959093921034528ecd6a59926e5fd543f56f94f2a0034ed4ba458c0a86
+size 19970698
diff --git a/tokenizer_config.json b/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..845872bf8f12ed0a158e912974476354ff69b287
--- /dev/null
+++ b/tokenizer_config.json
@@ -0,0 +1,330 @@
+{
+  "added_tokens_decoder": {
+    "151329": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151330": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151331": {
+      "content": "[gMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151332": {
+      "content": "[sMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151333": {
+      "content": "<sop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151334": {
+      "content": "<eop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151335": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151336": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151337": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151338": {
+      "content": "<|observation|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151339": {
+      "content": "<|begin_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151340": {
+      "content": "<|end_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151341": {
+      "content": "<|begin_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151342": {
+      "content": "<|end_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151343": {
+      "content": "<|begin_of_audio|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151344": {
+      "content": "<|end_of_audio|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151345": {
+      "content": "<|begin_of_transcription|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151346": {
+      "content": "<|end_of_transcription|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151347": {
+      "content": "<|code_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151348": {
+      "content": "<|code_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151349": {
+      "content": "<|code_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151350": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151351": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151352": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151353": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151354": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151355": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151356": {
+      "content": "<arg_key>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151357": {
+      "content": "</arg_key>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151358": {
+      "content": "<arg_value>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151359": {
+      "content": "</arg_value>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151360": {
+      "content": "/nothink",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151361": {
+      "content": "<|begin_of_box|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151362": {
+      "content": "<|end_of_box|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151363": {
+      "content": "<|image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151364": {
+      "content": "<|video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|image|>",
+    "<|video|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>",
+    "<|code_prefix|>",
+    "<|code_middle|>",
+    "<|code_suffix|>",
+    "/nothink"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "[MASK]",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": null,
+  "chat_template": "{# Unsloth template fixes #}\n[gMASK]<sop>\n{%- if tools -%}\n<|system|>\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>\n{% for tool in tools %}\n{{ tool | tojson|string }}\n{% endfor %}\n</tools>\n\nFor each function call, output the function name and arguments within the following XML format:\n<tool_call>{function-name}\n<arg_key>{arg-key-1}</arg_key>\n<arg_value>{arg-value-1}</arg_value>\n<arg_key>{arg-key-2}</arg_key>\n<arg_value>{arg-value-2}</arg_value>\n...\n</tool_call>{%- endif -%}\n{%- macro visible_text(content) -%}\n    {%- if content is string -%}\n        {{- content }}\n    {%- elif content is iterable and content is not mapping -%}\n        {%- for item in content -%}\n            {%- if item is mapping and item.type == 'text' -%}\n                {{- item.text }}\n            {%- elif item is mapping and (item.type == 'image' or 'image' in item) -%}\n                <|begin_of_image|><|image|><|end_of_image|>\n            {%- elif item is mapping and (item.type == 'video' or 'video' in item) -%}\n                <|begin_of_video|><|video|><|end_of_video|>\n            {%- elif item is string -%}\n                {{- item }}\n            {%- endif -%}\n        {%- endfor -%}\n    {%- else -%}\n        {{- content }}\n    {%- endif -%}\n{%- endmacro -%}\n{%- set ns = namespace(last_user_index=-1) %}\n{%- for m in messages %}\n    {%- if m.role == 'user' %}\n        {% set ns.last_user_index = loop.index0 -%}\n    {%- endif %}\n{%- endfor %}\n{% for m in messages %}\n{%- if m.role == 'user' -%}<|user|>\n{% if m.content is string %}\n{{ m.content }}\n{%- else %}\n{%- for item in m.content %}\n{% if item.type == 'video' or 'video' in item %}\n<|begin_of_video|><|video|><|end_of_video|>{% elif item.type == 'image' or 'image' in item %}\n<|begin_of_image|><|image|><|end_of_image|>{% elif item.type == 'text' %}\n{{ item.text }}\n{%- endif %}\n{%- endfor %}\n{%- endif %}\n{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith(\"/nothink\")) else '' -}}\n{%- elif m.role == 'assistant' -%}\n<|assistant|>\n{%- set reasoning_content = '' %}\n{%- set content = visible_text(m.content) %}\n{%- if m.reasoning_content is string %}\n    {%- set reasoning_content = m.reasoning_content %}\n{%- else %}\n    {%- if '</think>' in content %}\n        {%- set reasoning_content = ((content.split('</think>')|first).rstrip('\\n').split('<think>')|last).lstrip('\\n') %}\n        {%- set content = (content.split('</think>')|last).lstrip('\\n') %}\n    {%- endif %}\n{%- endif %}\n{%- if loop.index0 > ns.last_user_index and reasoning_content -%}\n{{ '\\n<think>' + reasoning_content.strip() +  '</think>'}}\n{%- else -%}\n{{ '\\n<think></think>' }}\n{%- endif -%}\n{%- if content.strip() -%}\n{{ '\\n' + content.strip() }}\n{%- endif -%}\n{% if m.tool_calls %}\n{% for tc in m.tool_calls %}\n{%- if tc.function %}\n    {%- set tc = tc.function %}\n{%- endif %}\n{{ '\\n<tool_call>' + tc.name }}\n{% set _args = tc.arguments %}{% if _args is mapping %}\n{% for k, v in _args|items %}\n<arg_key>{{ k }}</arg_key>\n<arg_value>{{ v | tojson|string if v is not string else v }}</arg_value>\n{% endfor %}{%- endif %}\n</tool_call>{% endfor %}\n{% endif %}\n{%- elif m.role == 'tool' -%}\n{%- if m.content is string -%}\n{%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n    {{- '<|observation|>' }}\n{%- endif %}\n{{- '\\n<tool_response>\\n' }}\n{{- m.content }}\n{{- '\\n</tool_response>' }}\n{% elif m.content is iterable and m.content is not mapping %}\n{%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n{{- '<|observation|>' }}\n{%- endif %}\n{{- '\\n<tool_response>\\n' }}\n{%- for tr in m.content -%}\n  {%- if tr is mapping and tr.type is defined -%}\n    {%- set t = tr.type | lower -%}\n    {%- if t == 'text' and tr.text is defined -%}\n{{ tr.text }}\n    {%- elif t in ['image', 'image_url'] -%}\n<|begin_of_image|><|image|><|end_of_image|>\n    {%- elif t in ['video', 'video_url'] -%}\n<|begin_of_video|><|video|><|end_of_video|>\n    {%- else -%}\n{{ tr | tojson|string }}\n    {%- endif -%}\n  {%- else -%}\n{{ tr.output if tr.output is defined else tr }}\n  {%- endif -%}\n{%- endfor -%}\n{{- '\\n</tool_response>' }}\n{%- else -%}\n<|observation|>{% for tr in m.content %}\n\n<tool_response>\n{{ tr.output if tr.output is defined else tr }}\n</tool_response>{% endfor -%}\n{% endif -%}\n{# ====== 逻辑结束 ====== #}\n{%- elif m.role == 'system' -%}\n<|system|>\n{{ visible_text(m.content) }}\n{%- endif -%}\n{%- endfor -%}\n{%- if add_generation_prompt -%}\n<|assistant|>\n{{'<think></think>\\n' if (enable_thinking is defined and not enable_thinking) else ''}}\n{%- endif -%}\n{# Copyright 2025-present Unsloth. Apache 2.0 License. #}"
+}
\ No newline at end of file
diff --git a/video_preprocessor_config.json b/video_preprocessor_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..52f4d0d1dfbc7a5ed58f2bfae105d9436cd9037a
--- /dev/null
+++ b/video_preprocessor_config.json
@@ -0,0 +1,11 @@
+{
+    "size": {"shortest_edge": 12544, "longest_edge": 47040000},
+    "do_rescale": true,
+    "patch_size": 14,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [0.48145466, 0.4578275, 0.40821073],
+    "image_std": [0.26862954, 0.26130258, 0.27577711],
+    "video_processor_type": "Glm46VVideoProcessor",
+    "processor_class": "Glm46VProcessor"
+}